Glass substrate micropore detection method and device and storage medium

Abstract

The invention discloses a glass substrate micropore detection method and device and a storage medium, and relates to the technical field of machine vision detection. The method mainly comprises the following steps: firstly, planning a double-platform motion track and an industrial camera image acquisition area, and obtaining a Mark point to calculate a rotation angle of a glass substrate as a basis for selecting a coordinate system transformation formula; secondly, planning a motion mode of the double platforms by taking time optimization as a principle, so that the double platforms cooperatively move in opposite directions to complete image acquisition of the whole glass substrate at a high speed; the method comprises the following steps: extracting a micropore mass center coordinate after multiple algorithm processing of an image, searching a matched coordinate point in a DXF template file after coordinate system transformation, finding out a miss-punched micropore coordinate after processing of all the images, and finally marking a leak hole in an original image. The problem that a large number of data samples and computing resources need to be used for long-time training of the model when a deep learning algorithm is adopted is solved, and the double-platform collaborative high-speed micropore detection system capable of being rapidly transplanted is provided and has the advantage of low cost.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and in particular to a method, a device and a storage medium for detecting micropores of a glass substrate. Background Art

[0002] Recently, Intel and Samsung released cutting-edge technology - glass substrate TGV (through-glass vias) technology, which is used in chip packaging. In the future, there is a trend to replace the chip packaging interposer and packaging substrate, achieve denser interconnection density between chips, and greatly reduce energy consumption, so as to better adapt to large chips such as data center AI. In the manufacturing process of glass substrates, microholes (non-through holes, approximate circular holes, usually with a circumscribed circle diameter of 5-10um and a spacing of 0.1-0.5mm) are first punched on the glass substrate with a laser, and then chemically etched to form an approximate through-hole circle. A single glass substrate generally has 40,000 to 60,000 microholes, and a glass substrate panel generally has more than 80,000 microholes. Due to the instability of laser equipment and other reasons, microholes are missed on the glass substrate. At this time, we need to check the missed microholes and obtain their coordinate positions, and then use lasers to punch the missed microholes again to ensure the function of the product. Normally, we use deep learning methods to detect leaks, but the use of deep learning algorithms requires a large number of data samples and computing resources to train the model for a long time. It is not possible to simply adjust the parameters and transplant the system and quickly put it into production to adapt to the current situation of rapid updates of glass substrate product models.

[0003] A glass substrate micropore detection method, device and storage medium of the present invention can effectively reduce the dependence on data samples, computing resources and training time, and invent a dual-platform collaborative high-speed micropore detection system that can be quickly transplanted, has high compatibility and low requirements on computing resources. Summary of the invention

[0004] The purpose of the present invention is to provide a glass substrate micropore detection method, device and storage medium. First, the motion trajectory of the dual-platform and the image acquisition area of ​​the industrial camera are planned, and the Mark point is obtained to calculate the rotation angle of the glass substrate as the basis for selecting the coordinate system transformation formula; then the motion mode of the dual-platform is planned based on the principle of time optimization, so that the two platforms move in opposite directions in coordination to complete the image acquisition of the entire glass substrate at a high speed; after multiple algorithm processing of the image, the centroid coordinates of the micropore are extracted, and then after the coordinate system transformation, the matching coordinate points are searched in the DXF template file. After all images are processed, the coordinates of the missed micropores can be found, and finally the missed holes are marked in the original image.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A glass substrate micropore detection device, characterized in that it comprises: a gantry motion platform, an auxiliary platform, a magnifying mirror, a zoom lens with a light source, an industrial camera, a coarse positioning fixture, two sets of lens brackets, a Z-axis assembly, an adjustment handle, a computer system, etc.; the gantry motion platform and the auxiliary platform can realize planar motion and the motion planes of the two are parallel;

[0007] The magnification of the light-emitting zoom lens is adjustable within a certain range, and the magnification of the expander is fixed; the expander, the light-emitting zoom lens, the lens barrel, etc. form an optical magnification system to achieve a multi-level optical magnification function; the optical magnification system is fixed on the Z-axis assembly through the two sets of lens brackets; the industrial camera is connected to the expander lens;

[0008] The Z-axis assembly is fixed on the X-axis mover of the gantry motion platform, and the adjustment handle enables the optical magnification system to move in the Z-axis direction through the slide rail-screw mechanism in the Z-axis assembly, thereby realizing the focus adjustment function of the industrial camera;

[0009] The thin pad is flat and wrinkle-free, opaque, has strong light reflection ability, and is not mirror-smooth; the glass substrate is placed on the thin pad, and the thin pad is placed on the auxiliary platform; the coarse positioning fixture performs positive tolerance positioning on the glass substrate;

[0010] A method for detecting micropores on a glass substrate, characterized by comprising: a device preparation stage, platform motion trajectory planning, micropore missed marking and re-inspection;

[0011] Preferably, the device preparation stage comprises the following steps:

[0012] S1: the coarse positioning fixture is opened, the glass substrate to be inspected is placed on the thin pad, and the coarse positioning fixture positions the glass substrate with a positive tolerance;

[0013] S2: setting the magnification of the zoom lens with light source according to the size of the micropore, setting the parameters of the industrial camera, and rotating the adjustment handle to obtain a uniformly illuminated and clear micropore image;

[0014] Preferably, the platform motion trajectory planning includes the following steps:

[0015] S3: The coarse positioning fixture positions the glass substrate with a positive tolerance and each photo has an overlapping area with the adjacent photo, so that the effective rectangular image size of the first row and first column of photos is longer than the center interval of adjacent photos in the same row, and wider than the center interval of adjacent photos in the same column; according to the field of view size determined by the industrial camera in step S2 and the size of the glass substrate, it is calculated that the photos in row a and column b are required to capture the image of the entire glass substrate;

[0016] S4: The industrial camera moves in an "S"-shaped trajectory and collects photos at fixed row and column intervals; compares the number of corners experienced from starting with "row" and "column" to collecting a complete glass substrate, and the solution with fewer corners is used as the motion trajectory of the industrial camera and the auxiliary platform;

[0017] S5: Mark point image acquisition: establish a world coordinate system and its motion origin for the industrial camera and the auxiliary platform, record the position of the industrial camera in the world coordinate system when acquiring the Mark point image, and extract the pixel coordinates of the three Mark points;

[0018] S6: Rotation angle calculation: The actual position of the glass substrate on the auxiliary platform is regarded as the position translation and rotation of the DXF template file. A coordinate system is established in the glass substrate micro-hole processing DXF template file. The DXF template file is parsed to obtain the coordinates and radius of the micro-holes, which are stored in the program matrix mat. The rotation angle of the glass substrate is calculated:

[0019] S7: Motion trajectory and method planning of image acquisition: obtain the first row and first column image, then, when starting the movement in the row / column direction:

[0020] Eg1: When the interval E / D between the centers of adjacent photos in the same row / column is large, in the row / column direction, at the non-photographing position, the industrial camera and the auxiliary platform move toward each other in coordination according to the planned speed and trajectory; at the photographing position, one of the industrial camera and the auxiliary platform slows down to low uniform linear motion, and the other stops moving when it reaches the photographing position, or both slow down to low uniform coordinated motion toward each other to collect images;

[0021] Eg2: When the center interval E / D of adjacent photos in the same row / column is small, in the row / column direction, the industrial camera and the auxiliary platform adopt a uniform linear motion mode throughout the entire process (except the corner of the track), and move to the corresponding position according to the planned track to collect images;

[0022] At the corner of the track, the industrial camera and the auxiliary platform adopt a coordinated movement toward each other, stop moving when reaching the target position, and then collect images;

[0023] The industrial camera and the auxiliary platform move toward each other in coordination. Between two adjacent image acquisition positions, they move together in the column direction by D = ΔY + ΔY 1 , or move together in the row direction by E = ΔX + ΔX 1 , where D and E are constants, ΔY and ΔY 1 , ΔX, ΔX 1 Random value, depending on the real-time collaborative movement planning according to the principle of time optimization; when taking pictures, the absolute coordinates of the industrial camera and the auxiliary platform are obtained as X, Y and X1 , Y 1 ,calculate

[0024]

[0025] Where int is the integer function, round is the rounding function, M is the image label collected at this position and is combined with X, Y, X 1 , Y 1 Values ​​and images are stored in bundles.

[0026] Preferably, the micropore missed marking and re-inspection comprises the following steps:

[0027] S8: Image processing and coordinate transformation:

[0028] S81: The image is Gaussian filtered and binarized, and then the connectedComponentsWithStats function is used to obtain all connected domains S and their parameters. min ≤s i ≤s max Screen out the connected domain S that matches the actual micropores i ;calculate According to the z value, determine the rotation direction of the glass substrate and the positive or negative corresponding sinθ value;

[0029] S82: Put S i The corresponding centroid coordinates of each micropore connected domain are converted from the pixel coordinate system μo 0 υ is transformed into the xoy coordinate system, and we get (x` i ,y` i ); Then the centroid of each microporous connected domain (x` i ,y` i ) in the matrix mat by satisfying the constraints |x` i -x|<δ and |y` i -y|<δFind the coordinate point (x k ,y k ), (x` i ,y` i ) matches a coordinate point (x k ,y k ), put (x k ,y k ) is stored in the program matrix mat1 and stops immediately (x i ,y i ,) and the remaining coordinates of the matrix mat continue to match; in order to make the particle (x` i ,y` i ) Find no more than one matching point in the matrix mat and pair it with the closest coordinate point, δ must satisfy δ<0.5L min, L min is the shortest distance between any two circle centers in the DXF template file; after all images are collected, processed, and coordinate transformed and matched in the above manner, the coordinates of the missing points (x j ,y j );

[0030] S9: Micro-hole missing mark and re-inspection:

[0031] S91:(x j ,y j ) into the following formula

[0032]

[0033] Find the same number "M" from "m" and get the leak point (x j ,y j ) in the image labeled “M”, where trunc is the truncation function;

[0034] S92: In the image labeled “M”, the xoy coordinate system point (x j ,y j ) First, transform it to the pixel coordinate system μo through inverse translation and rotation transformation, and then transform it to the pixel coordinate system μo through coordinate system transformation 0 υ, the pixel coordinates (μ j ,υ j ), and then draw a circle with the pixel coordinate point as the center and a radius of R, marking the position of the missed micro-hole on the image and displaying it on the display screen;

[0035] S10: Develop a re-inspection system. When the position of the missed micro-hole is displayed on the screen, the system prompts "Confirm whether the micro-hole is missed". After manual or other judgment methods confirm, the answer is "yes", then the image is renamed with "m", and the renamed and marked image is compared with (x j ,y j )Bundle storage; if you answer "No", it will not be stored (x j ,y j ) and image; the re-inspection system can be shielded. After shielding, the system will no longer prompt "Confirm whether micro-holes are missed", and the program will directly enter to rename the image with "m" and compare the renamed and marked image with (x j ,y j ) bundled storage. After completing the missed micro-hole detection on the glass substrate, only the missed micro-hole coordinates (x j ,y j ) and its marked image, the two are stored together, and other detection process data are deleted. If there are multiple leaks in one image, they should be uniformly marked in this image.

[0036] The computer system comprises: at least one processor, at least one memory and computer program instructions stored in the memory, and when the computer program instructions are executed by the processor, the method described above is implemented.

[0037] A glass substrate micropore detection storage medium, characterized in that a storage medium stores computer program instructions thereon, characterized in that when the computer program instructions are executed by a processor, the above-mentioned method is implemented.

[0038] It can be seen from the above technical solutions that the present invention has the following advantages: The present invention discloses a dual-platform collaborative high-speed micropore detection system that can be quickly transplanted, which has the advantages of adapting to fast product updates, visualization of detection results, and low cost. The detection system overcomes the problem that a large number of data samples and computing resources are required for long-term training of the model when using deep learning algorithms, and the application cannot be quickly transplanted. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of a glass substrate micropore detection device provided by an embodiment of the present invention.

[0040] Figure 2 A schematic diagram of a method for image acquisition and platform motion trajectory planning for micropore detection on a glass substrate provided in an embodiment of the present invention.

[0041] Figure 3 A schematic diagram of a DXF template file for micro-hole processing on a glass substrate provided in an embodiment of the present invention.

[0042] Figure 4 A schematic diagram of the translation and rotation of a glass substrate provided by an embodiment of the present invention.

[0043] Figure 5 A diagram illustrating the positioning principle of a micro-hole on a glass substrate provided in an embodiment of the present invention.

[0044] Figure 6 A flow chart of a method for detecting micropores in a glass substrate provided by an embodiment of the present invention.

[0045] in:

[0046] 1Gantry motion platform 2Auxiliary platform

[0047] 3 magnifying mirror 4 magnification mirror with light source

[0048] 5Industrial camera 6Coarse positioning fixture

[0049] 7-lens bracket 8-lens bracket

[0050] 9Z axis assembly 10 adjustment handle

[0051] 11 Computer system 12 X axis

[0052] 13 thin pad 14 glass substrate

[0053] 15 Y axis H glass substrate length

[0054] L Glass substrate width DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0056] Glass substrate TGV (through-glass vias) technology uses lasers to punch tens of thousands of microholes on glass substrates. Due to the instability of laser equipment and other reasons, microholes are missed on the glass substrate. During the production process, we need to check the missed microholes and obtain their coordinate positions, and then use lasers to re-punch the missed microholes to ensure the product of this process. Usually, we use deep learning methods to detect leaks, but the use of deep learning algorithms requires a large number of data samples and computing resources for long-term training of the model. It is not possible to simply adjust the parameters and transplant the model to quickly put it into production to adapt to the current situation of rapid updates of glass substrate product models.

[0057] A method, device and storage medium for detecting micropores on a glass substrate of the present invention use a coordinate point matching method to find leaks, which requires fewer computing resources than a deep learning algorithm based on an image template matching method. When parameters such as the glass substrate size, micropore size, micropore spacing, and micropore arrangement change, it is only necessary to change the corresponding values ​​of the parameters in the program so that the detection system can quickly adapt to the production of new products. It has the characteristics of strong compatibility, which is different from the deep learning algorithm, which requires a large number of image samples to be re-collected for a long time to train the model before it can be put into the production of new products. In view of the problem that it is difficult for the motion platform to obtain a high motion speed within a short stroke, the image acquisition efficiency is improved by using a dual-platform opposite motion method. In summary, a method, device and storage medium for detecting micropores on a glass substrate of the present invention do not rely on a large number of data samples and a long-term training model, and provide a dual-platform collaborative high-speed micropore detection system that can be quickly transplanted using conventional computing resources, which has obvious low-cost advantages.

[0058] like Figure 1As shown, the main structure of a glass substrate micropore detection device includes: a gantry motion platform 1, an auxiliary platform 2, a magnifying lens 3, a zoom lens with a light source 4, an industrial camera 5, a coarse positioning fixture 6, a lens bracket 7, a lens bracket 8, a Z-axis component 9, an adjustment handle 10, a computer system 11, etc.; the gantry motion platform 1 and the auxiliary platform 2 can realize planar motion and the motion planes of the two are parallel, so as to ensure the imaging quality of the industrial camera.

[0059] The magnification of the light-emitting zoom lens 4 is adjustable within a certain range, and is used to be compatible with micro-hole imaging of different sizes; the magnification of the expander 3 is fixed, and it mainly serves to increase the magnification of the optical magnification system; the expander 3, the light-emitting zoom lens 4, the lens barrel, etc. constitute the optical magnification system to achieve a multi-stage optical magnification function; the optical magnification system is fixed on the Z-axis assembly 9 through the lens bracket 7 and the lens bracket 8; the industrial camera 5 is connected to the lens of the expander 3.

[0060] The Z-axis assembly 9 is fixed on the X-axis 12 mover of the gantry motion platform 1, so as to realize the synchronous movement of the X-axis 12 mover of the gantry motion platform 1 and the Z-axis assembly 9. The adjustment handle 10 enables the optical magnification system to move in the Z-axis direction through the slide rail-screw mechanism in the Z-axis assembly 9, so as to realize the focus adjustment function of the industrial camera 5.

[0061] The thin pad 13 is flat and wrinkle-free, opaque, has strong light reflection ability, and is not mirror-smooth, so that the micropores and the non-perforated surface on the glass substrate 14 form two dark and bright areas. The glass substrate 14 is placed on the thin pad 13, and the thin pad 13 is placed on the auxiliary platform 2. The rough positioning fixture 6 performs positive tolerance positioning on the glass substrate 14; positioning, on the one hand, reduces the rotation angle of the glass substrate 14, thereby reducing the actual area of ​​the glass substrate 14 that the industrial camera 5 needs to collect, and on the other hand, paves the way for leak hole positioning; positive tolerance positioning prevents the glass substrate 14 from being crushed by squeezing.

[0062] The glass substrate micropore detection method includes three steps: device preparation stage, platform motion trajectory planning, micropore missed marking and re-inspection.

[0063] The device preparation stage includes the following steps:

[0064] S1: the coarse positioning fixture 6 is opened, the glass substrate 14 to be inspected is placed on the thin pad 13, and the coarse positioning fixture 6 performs positive tolerance positioning on the glass substrate 14;

[0065] S2: Set the magnification of the light source zoom lens 4 according to the size of the micropore, set the parameters of the industrial camera 5, and rotate the adjustment handle 10 to obtain a uniformly illuminated and clear micropore image. When using it for the first time or changing the product, the industrial camera 5 needs to be calibrated first and the pixel scale K is obtained.

[0066] Among them, the platform motion trajectory planning includes the following steps:

[0067] S3: Figure 2 As shown, the coarse positioning fixture 6 positions the glass substrate 14 with a positive tolerance and each photo has an overlapping area ΔH*ΔL with the adjacent photo, so that the effective rectangular image size of the first row and first column of photos has a length F slightly larger than the center interval E of adjacent photos in the same row, and a width G slightly larger than the center interval D of adjacent photos in the same column. According to the field of view size determined by the industrial camera 5 in step S2 and the size H*L of the glass substrate 14, it is calculated that a row and b columns of photos are required to capture the image of the entire glass substrate 14.

[0068] S4: The industrial camera 5 moves in an "S"-shaped trajectory and collects photos at fixed row and column intervals; compares the number of corners experienced from starting with "row" and "column" to collecting the entire glass substrate 14, and the solution with fewer corners is used as the movement trajectory of the industrial camera 5 and the auxiliary platform 2.

[0069] S5: Mark point image acquisition: Create the following for industrial camera 5 and auxiliary platform 2: Figure 1 The world coordinate system O-XYZ and OX 1 Y 1 Z 1 .like Figure 2 and Figure 3 As shown, A and P 0 , B and P m It is the same point in different coordinate systems. The image acquisition position of the first group of Mark points is the photo acquisition position of the first row and first column. In this case, this position is also the starting point of image acquisition O 0 At this time, the gantry motion platform 1, auxiliary platform 2 and industrial camera 5 are at the origin, and the pixel coordinates of the Mark point are extracted as The image acquisition position of the second group of Mark points is the photo acquisition position of the ath row and the bth column. In this case, this position is also the end position of the image acquisition. m , extract the pixel coordinates of the Mark point as The third group of Mark point P n It is used as a backup when the two sets of Mark points cannot uniquely determine the state of the glass substrate 14. 0 Movement to O m , moved in the positive direction of the X axis by X m , moved Y in the positive direction of the Y axis m , records the position of industrial camera 5 in the world coordinate system when it collects the Mark point image.

[0070] S6: Rotation angle calculation: Figure 4 As shown, the actual position of the glass substrate 14 on the auxiliary platform 2 is regarded as the position translation and rotation of the DXF template file, and the micro-hole processing DXF template file of the glass substrate 14 is established as follows: Figure 3 The xoy coordinate system shown in the figure parses the DXF template file to obtain the coordinates (x, y) and radius r of the microhole, which are stored in the matrix mat in the program. On the xoy plane, A=(0,0), B=(x B ,y B ), The pixel coordinates P 0 , P m Transformed into the xoy plane coordinate system, we have: Where K is the pixel scale (mm / pixl). Calculate the glass substrate rotation angle:

[0071] The micropores are small in size and need to be optically magnified before they can be processed by the visual algorithm. However, the field of view of the industrial camera 5 is small, and hundreds or thousands of photos are usually required to capture the image of the complete glass substrate 14. In order to improve the efficiency of image acquisition, a common method is to increase the platform movement speed. However, the field of view of the industrial camera 5 is small, and it is difficult to obtain a high movement speed within a short stroke. To address this problem, the present invention adopts two methods to improve the image acquisition efficiency. One is to use Figure 2 The first is a method in which the two platforms move towards each other, and the second is a method in which at least one of the moving platforms does not stop moving when taking pictures.

[0072] S7: Motion trajectory and method planning of image acquisition: first acquire the image located in the first row and the first column. When step S4 determines that the industrial camera 5 starts to move from the column as the starting point, Figure 2 As shown in:

[0073] Eg1: When the center distance between adjacent photos in the same column is D = ΔY + ΔY 1 When it is larger: within the center interval D, the industrial camera 5 and the auxiliary platform 2 can obtain a movement speed that is greater than the maximum speed v when the industrial camera 5 takes pictures without generating a smear under the condition of uniform linear motion. max At this time, in the column direction, at the non-photographing position, the industrial camera 5 and the auxiliary platform 2 move toward each other in coordination according to the planned speed and trajectory; at the photographing position, the industrial camera 5 and the auxiliary platform 2 move down to a low uniform straight line (speed is ν max ) movement, the other party reaches the photo taking position and stops moving, or both parties slow down to a low uniform speed (speed is 0.5ν max )Collect images by coordinated opposite motion.

[0074] Eg2: When the center distance between adjacent photos in the same column is D = ΔY + ΔY 1 When smaller: the motion speed obtained by the industrial camera 5 and the auxiliary platform 2 is less than the maximum speed ν of the industrial camera 5 when taking pictures without smear under the condition of uniform linear motion maxAt this time, in the column direction, the industrial camera 5 and the auxiliary platform 2 use a uniform straight line (speed of 0.5v) throughout the entire process (except the corner of the track). max ) move towards each other and move collaboratively to the corresponding position according to the planned trajectory to collect images.

[0075] When step S4 determines that the industrial camera 5 starts to move with the action start, similarly, the motion trajectory and method of image acquisition are planned according to the above two situations Eg1 and Eg2.

[0076] At the corner of the track, the industrial camera 5 and the auxiliary platform 2 both adopt a coordinated movement mode toward each other, stop moving when reaching the target position, and then collect images.

[0077] The industrial camera 5 and the auxiliary platform 2 move together. At two adjacent image acquisition positions, the industrial camera 5 and the auxiliary platform 2 move together in the column direction by D = ΔY + ΔY 1 Or they move together in the row direction by E = ΔX + ΔX 1 , where D and E are the fixed values ​​planned in step S4, ΔY, ΔX, ΔY 1 , ΔX 1 Random values, representing the movement of the industrial camera 5 in the Y axis 15 and X axis 12 directions, and the auxiliary platform 2 in the Y 1 Axis, X 1 The amount of movement in the axis direction depends on the real-time collaborative motion planning based on the principle of time optimization; when taking pictures, the absolute coordinates of the industrial camera 5 in the world coordinate system O-XYZ are obtained, which are X, Y and the absolute coordinates of the auxiliary platform 2 in the world coordinate system OX 1 Y 1 Z 1 Absolute coordinates and X 1 , Y 1 ,calculate

[0078]

[0079] Where int is the integer function, round is the rounding function, M is the image label collected at this position and is combined with X, Y, X 1 , Y 1 Values ​​and images are stored in bundles.

[0080] Among them, micro-hole missed marking and re-inspection include the following steps:

[0081] S8: Image processing and coordinate transformation:

[0082] S81: The image is Gaussian filtered and binarized, and then the connectedComponentsWithStats function is used to obtain all connected domains S and their parameters. min ≤s i≤s max Screen out the connected domain S that matches the actual micropores i ;calculate Figure 1 O-XYZ is a left-hand coordinate system. If z>0, according to the left-hand rule, the glass substrate rotates clockwise. If z<0, according to the left-hand rule, the glass substrate rotates counterclockwise.

[0083] S82: Put S i The corresponding centroid coordinates of each micropore connected domain are converted from the pixel coordinate system μo 0 υ is transformed into the xoy coordinate system, and we get (x` i ,y` i ); Then the centroid of each microporous connected domain (x` i ,y` i ) in the matrix mat by satisfying the constraints |x` i -x|<δ and |y` i -y|<δFind the coordinate point (x k ,y k ), (x` i ,y` i ) matches a coordinate point (x k ,y k ), put (x k ,y k ) is stored in the program matrix mat1 and stops immediately (x` i ,y` i ) continues to match the remaining coordinates of the matrix mat; in order to make the particle (x` i ,y` i ) Find no more than one matching point in the matrix mat and pair it with the closest coordinate point, δ must satisfy δ<0.5L min , L min is the shortest distance between any two circle centers in the DXF template file. After all images are collected, processed, and coordinate transformed and matched in the above manner, the coordinates of the center of the leak hole (x j ,y j ).

[0084] In step S82, the leak is found by coordinate point matching, which requires less computing resources than the traditional deep learning algorithm based on image template matching.

[0085] Since each glass substrate 14 is placed at a different position on the thin pad 13, it is impossible to realize that each image corresponds to a small area DXF template file. Therefore, in step S82, after all images are collected, processed, and coordinate transformed and matched in the above manner, the matrix mat and the matrix mat1 are compared to obtain the coordinates of the center of the leak hole (x j ,y j In order to reduce the demand for computing resources, the coordinates of the particles corresponding to the connected domain Si (x i ,y i ,) Coordinate of the center of the leak hole (x j ,y j ) to locate the original image and finally identify the leak. To solve this problem, the following method with less computing resource requirements is adopted, using the following two conditions:

[0086] (a) Each time an image is captured, the industrial camera 5 and the auxiliary platform 2 move together with intervals D and E fixed.

[0087] (b) The glass substrate 14 is positioned with positive tolerance so that the effective rectangular image size F*G of the first row and first column of photos has a length F slightly larger than the center interval E of adjacent photos in the same row, and a width G slightly larger than the center interval D of adjacent photos in the same column. Each photo has an overlapping area with the adjacent photo.

[0088] Each photo has an overlapping area with the adjacent photo. One of the purposes is to prevent the appearance of some micro-hole images at the junction of the photos, where part of the image is in one photo and the other part is in another photo. In subsequent image processing, the connected domain area s min ≤s i ≤s max Screen out those that are mistakenly identified as leaks. Another purpose is to construct condition (b).

[0089] like Figure 5 As shown, the above two prerequisites make the micro-hole center coordinate point (x n ,y n )When 0≤x n When ≤E, x n falls within the range of F, which is also within the range of the first column of photos; when E≤x n When ≤2E, x n Falling in F 1 range, that is, within the range of the second column of photos; when 2E≤x n When ≤3E, x n Falling in F 2 range, that is, within the range of the third column of photos; similarly, when 0≤y n ≤D, y n It falls within the range of G, which is also within the range of the first row of photos; when D≤yn When ≤2D, y n Falling in G 1 range, that is, within the range of the second column of photos; when 2D≤y n When ≤3D, y n Falling in G 2 range, that is, within the range of the third column of photos; other ranges x n ,y n There are also the above rules, where F 1 =F 2 =F 3 =…,G 1 =G 2 =G 3 =… Therefore, we use this rule to know the coordinate point of the leak hole (x j ,y j ), calculate x j / E,,y j / D ratio can tell the corresponding microporous connected domain centroid (x` j ,y` j ) in the row and column of the photo. Similarly, for the world coordinate system O-XYZ and the world coordinate system OX 1 Y 1 Z 1 The lower coordinate and (X n ,Y n ) also has the same above rule, where X n =|X 1 |+|X|,Y n =|Y 1 |+|Y|. The specific method is as follows:

[0090] S9: Micro-hole missing mark and re-inspection:

[0091] S91:(x j ,y j ) into the following formula

[0092]

[0093] Find the same number "M" from "m" and get the leak point (x j ,y j ) in the image labeled “m”, where trunc is the truncation function.

[0094] In the above, formula (1) uses the coordinate positions of the industrial camera 5 and the auxiliary platform 2 when the image is collected to encode the collected image with four digits. Part ① of formula (1) is to obtain the first two digits representing the column of photos, and part ② is to obtain the last two digits representing the row of photos. Because there is ambiguity in the interpretation of the meaning of three or less digits, for example, the code "921" can be translated into "09, 21" and "92, 01" using four digits, corresponding to two images. Formula (1) and formula (2) "+10" makes the code start from "1010", at which time a four-digit number uniquely corresponds to an image. If the number of photos is large, more digits can be considered for encoding.

[0095] S92: In the image labeled “M”, the xoy coordinate system point (x j ,y j ) First, transform it to the pixel coordinate system μo through inverse translation and rotation transformation, and then transform it to the pixel coordinate system μo through coordinate system transformation 0 υ, the pixel coordinates (μ j ,υ j ), and then draw a circle with the pixel coordinate point as the center and radius R, mark the position of the missed micro-hole on the image and display it on the display screen.

[0096] In formula (2), parts ④ and ⑥ convert the xoy coordinate system point (x j ,y j ) is transformed by inverse translation and rotation to obtain the corresponding (x` j ,y` j ), (x` j ,y` j )Through parts ③ and ⑤, the image with the same label as “m” is located.

[0097] S10: Develop a re-inspection system to manually re-inspect or spot-check the correctness of the test results of the glass substrate inspection system. When the position of the missed micro-hole is displayed on the screen, the system prompts "Confirm whether the micro-hole is missed". After manual or other judgment methods confirm, answer "yes", then rename the image with "m" and compare the renamed and marked image with (x j ,y j )Bundle storage; if you answer "No", it will not be stored (x j ,y j ) and image; the re-inspection system can be shielded. After shielding, the system will no longer prompt "Confirm whether micro-holes are missed", and the program will directly enter to rename the image with "m" and compare the renamed and marked image with (x j ,y j )Bundle storage.

[0098] In order to save computer storage resources, after completing the missed micro-hole detection of the glass substrate 14, only the coordinates of the missed micro-holes and the marked images are saved, and the two are stored together, and other detection process data are deleted. If there are multiple missed holes in an image, they should be uniformly marked in this image. Figure 6 shown.

[0099] The computer system 11 includes: at least one processor, at least one memory, and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, the above method is implemented.

[0100] A glass substrate micropore detection storage medium, a storage medium, on which computer program instructions are stored, when the computer program instructions are executed by a processor, the above method is implemented.

[0101] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A glass substrate micropore detection device, characterized in that: include: Gantry motion platform, auxiliary platform, magnifying mirror, zoom mirror with light source, industrial camera, coarse positioning fixture, two sets of lens brackets, Z-axis assembly, adjustment handle, computer system, etc.; the gantry motion platform and the auxiliary platform can realize planar motion and the motion planes of the two are parallel; The magnification of the light-emitting zoom lens is adjustable within a certain range, and the magnification of the expander is fixed; the expander, the light-emitting zoom lens, the lens barrel, etc. constitute an optical magnification system; the optical magnification system is fixed on the Z-axis assembly through the two sets of lens brackets; the industrial camera is connected to the expander lens; The Z-axis assembly is fixed on the X-axis mover of the gantry motion platform, and the adjustment handle enables the optical magnification system to move in the Z-axis direction through the slide rail-screw mechanism in the Z-axis assembly, thereby realizing the focus adjustment function of the industrial camera; The thin pad is flat and wrinkle-free, opaque, has strong light reflection ability, and is non-mirror smooth; the glass substrate is placed on the thin pad, and the thin pad is placed on the auxiliary platform; the rough positioning fixture performs positive tolerance positioning on the glass substrate.

2. A method for detecting micropores in a glass substrate, characterized in that: include: Device preparation stage, platform motion trajectory planning, micro-hole missed marking and re-inspection.

3. The method for detecting micropores in a glass substrate according to claim 2, characterized in that: The device preparation stage includes the following steps: S1: the coarse positioning fixture is opened, the glass substrate to be inspected is placed on the thin pad, and the coarse positioning fixture positions the glass substrate with a positive tolerance; S2: According to the size of the micropore, the magnification of the zoom lens with light source is set, the parameters of the industrial camera are set, and the adjustment handle is turned to obtain a micropore image with uniform illumination and clarity.

4. The method for detecting micropores in a glass substrate according to claim 2, characterized in that: The platform motion trajectory planning includes the following steps: S3: The coarse positioning fixture positions the glass substrate with a positive tolerance and each photo has an overlapping area with the adjacent photo, so that the effective rectangular image size of the first row and first column of photos is slightly larger in length than the center interval of adjacent photos in the same row, and slightly larger in width than the center interval of adjacent photos in the same column; according to the field of view size determined by the industrial camera in step S2 and the size of the glass substrate, it is calculated that the number of photos in row a and column b is required to capture the image of the entire glass substrate; S4: the industrial camera moves in an "S"-shaped trajectory, and collects photos at fixed row and column intervals; compares the number of corners experienced from starting with "row" and "column" to collecting a complete glass substrate, and the solution with fewer corners is used as the movement trajectory of the industrial camera and the auxiliary platform; S5: Mark point image acquisition: establish a world coordinate system and its motion origin for the industrial camera and the auxiliary platform, record the position of the industrial camera in the world coordinate system when acquiring the Mark point image, and extract the pixel coordinates of the three Mark points; S6: Rotation angle calculation: The actual position of the glass substrate on the auxiliary platform is regarded as the position translation and rotation of the DXF template file. A coordinate system is established in the glass substrate micro-hole processing DXF template file. The DXF template file is parsed to obtain the center coordinates and radius of the micro-hole, which are stored in the program matrix mat. The rotation angle of the glass substrate is calculated: S7: Motion trajectory and method planning of image acquisition: obtain the first row and first column image, then, when starting the movement in the row / column direction: Eg1: When the interval E / D between the centers of adjacent photos in the same row / column is large, in the row / column direction, at the non-photographing position, the industrial camera and the auxiliary platform move toward each other in coordination at the planned speed and trajectory; At the photographing position, one of the industrial camera and the auxiliary platform is reduced to a low uniform linear motion, and the other stops moving when it reaches the photographing position, or both are reduced to a low uniform speed and move in a coordinated manner toward each other to collect images; Eg2: When the center interval E / D of adjacent photos in the same row / column is small, in the row / column direction, the industrial camera and the auxiliary platform adopt a uniform linear motion mode throughout the entire process (except the corner of the track), and move to the corresponding position according to the planned track to collect images; At the corner of the track, the industrial camera and the auxiliary platform adopt a coordinated movement toward each other, stop moving when reaching the target position, and then collect images; The industrial camera and the auxiliary platform move toward each other in coordination. Between two adjacent image acquisition positions, they move together in the column direction by D=ΔY+ΔY1, or in the row direction by E=ΔX+ΔX1, where D and E are fixed values, and ΔY, ΔY1, ΔX, and ΔX1 are random values, depending on whether the real-time coordinated movement is planned according to the principle of time optimization. When taking pictures, the absolute coordinates of the industrial camera and the auxiliary platform are calculated as X, Y and X1, Y1 respectively. Where int is the integer function, round is the rounding function, and M is the image label collected at this position and is stored in bundles with the X, Y, X1, Y1 values ​​and the image.

5. The method for detecting micropores in a glass substrate according to claim 2, characterized in that: The micropore missed marking and re-inspection comprises the following steps: S8: Image processing and coordinate transformation: S81: The image is Gaussian filtered and binarized, and then the connectedComponentsWithStats function is used to obtain all connected domains S and their parameters. min ≤s i ≤s max Screen out the connected domain S that matches the actual micropores i ;calculate According to the z value, determine the rotation direction of the glass substrate and the positive or negative corresponding sinθ value; S82: Put S i The corresponding centroid coordinates of each micropore connected domain are transformed from the pixel coordinate system μo0υ to the xoy coordinate system, and (x` i ,y` i ); Then the centroid of each microporous connected domain (x` i ,y` i )In the matrix mat, according to the constraints |x` i -x|<δ and |y` i -y|<δFind the coordinate point (x k ,y k ), (x` i ,y` i ) matches a coordinate point (x k ,y k ), put (x k ,y k ) is stored in the program matrix mat1 and stops immediately (x` i ,y` i ) continues to match the remaining coordinates of the matrix mat; δ must satisfy δ<0.5L min , L min is the shortest distance between any two circle centers in the DXF template file; after all images are collected, processed, and coordinate transformed and matched in the above manner, the coordinates of the missing points (x j ,y j ); S9: Micro-hole missing mark and re-inspection: S91:(x j ,y j ) into the following formula Find the same number "M" from "m" and get the leak point (x j ,y j ) in the image labeled "M", where trunc is the truncation function; S92: In the image labeled "M", the xoy coordinate system point (x j ,y j ) is first transformed into the pixel coordinate system μo0υ through inverse translation and rotation transformation, and then through coordinate system transformation, the pixel coordinates (μo0υ) in the image can be calculated. j ,υ j ), and then draw a circle with the pixel coordinate point as the center and a radius of R, marking the position of the missed micro-hole on the image and displaying it on the display screen; S10: Develop a re-inspection system. When the position of the missed micro-hole is displayed on the screen, the system prompts "Confirm whether the micro-hole is missed". After confirmation by manual or other judgment methods, the answer is "yes", then the image is renamed with "m" and the renamed and marked image is compared with (x j ,y j ) bundle storage; if you answer "no", it will not be stored (x j ,y j ) and image; the re-inspection system can be shielded. After shielding, the system will no longer prompt "Confirm whether micro-holes are missed", and the program will directly enter to rename the image with "m" and compare the renamed and marked image with (x j ,y j ) bundled storage; after completing the missed micro-hole detection of the glass substrate, only the missed micro-hole coordinates (x j ,y j ) and its marked image, the two are stored together, and other detection process data are deleted; if there are multiple leaks in one image, they should be uniformly marked in this image.

6. The glass substrate micropore detection device according to claim 1, characterized in that: The computer system comprises: at least one processor, at least one memory, and computer program instructions stored in the memory, and when the computer program instructions are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

7. A glass substrate micropore detection storage medium, characterized in that: A storage medium having computer program instructions stored thereon, characterized in that when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images