Glass substrate micropore detection method and device and storage medium

Abstract

The application discloses a kind of glass substrate micropore detection method, device and storage medium, it is related to machine vision detection technical field;Mainly include the following steps:First, plan double platform motion trajectory and industrial camera image acquisition area, obtain Mark point calculation glass substrate rotation angle, as the basis for selecting coordinate system conversion formula;Then with time optimization as principle plan double platform motion mode, so that the two collaborative movement high speed completes whole glass substrate image acquisition;Image multiple algorithm processing extracts micropore centroid coordinate, then after coordinate system conversion finds matching coordinate point in DXF template file, after all image processing, can find the coordinate of the missed micropore, finally mark the hole in the original image.The application overcomes the problem that a large number of data samples and computing resources are required for long-time training of the model when using a deep learning algorithm, and provides a double-platform collaborative high-speed micropore detection system that can be quickly transplanted, with a low-cost advantage.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision detection, and in particular to a glass substrate micro-hole detection method, device and storage medium. BACKGROUND

[0002] Recently, Intel and Samsung have released a cutting-edge technology-glass substrate TGV (through-glass vias) technology, which is applied in chip packaging. In the future, it has the trend of replacing the interlayer and packaging substrate in chip packaging, realizing more intensive interconnection density between chips, greatly reducing energy consumption, and better adapting to large chips such as data center AI. In the glass substrate manufacturing process, a micro-hole (non-through hole, approximately circular hole, usually with an outer circle diameter of 5-10 um, and a spacing between holes of 0.1-0.5 mm) is first punched on the glass substrate by laser, and then an approximately through-hole circle is formed by chemical corrosion. A single glass substrate generally has 40-60 thousand micro-holes, and a glass substrate panel generally has more than 80 thousand micro-holes. Due to unstable laser equipment and other reasons, micro-holes are missed on the glass substrate. At this time, we need to check out the missed micro-holes and obtain their coordinate positions, and then use laser to re-punch the missed micro-holes to ensure the function of the product. Under normal circumstances, we will use a deep learning method to detect the missing holes, but when using a deep learning algorithm, a large amount of data samples, computing resources, and long training time are required to train the model. It cannot be transplanted to the system after simple parameter adjustment and quickly put into production to adapt to the current situation of fast updating of glass substrate product models.

[0003] The glass substrate micro-hole detection method, device and storage medium of the present application can effectively reduce the dependence on data samples, computing resources and training time, and a dual-platform collaborative high-speed micro-hole detection system with fast transplantation, high compatibility and low computing resource requirement is invented. SUMMARY

[0004] The purpose of the present application is to provide a glass substrate micro-hole detection method, device and storage medium. First, the motion trajectory of the dual platform and the image acquisition area of the industrial camera are planned to obtain the Mark point calculation glass substrate rotation angle as the basis for selecting the coordinate system transformation formula. Then, the motion mode of the dual platform is planned based on the principle of time optimization, so that the two platforms move towards each other at high speed to complete the image acquisition of the whole glass substrate. After multiple image processing algorithms, the micro-hole centroid coordinates are extracted, and then the matching coordinate points are found in the DXF template file after coordinate system transformation. After all image processing is completed, the missed micro-hole coordinates can be found, and finally the missing holes are marked in the original image.

[0005] The purpose of the present application can be achieved by the following technical solutions:

[0006] A kind of glass substrate micropore detection device, characterized in that, including: gantry motion platform, auxiliary platform, expansion mirror, with light source variable power lens, industrial camera, coarse positioning fixture, two groups of lens holder, Z-axis component, adjusting handle, computer system etc.;The gantry motion platform with the auxiliary platform can realize plane motion and the motion plane of both is parallel;

[0007] The amplification of the with light source variable power lens is adjustable within a certain range, and the amplification of the expansion mirror is fixed;The expansion mirror, with light source variable power lens, lens barrel and the like constitute an optical magnification system, realizing multi-stage optical magnification function;The optical magnification system is fixed on the Z-axis component through the two groups of lens holder;The industrial camera is connected with the expansion mirror lens;

[0008] The Z-axis component is fixed on the gantry motion platform X-axis mover, and the adjusting handle moves the optical magnification system in the Z-axis direction through the slide rail-screw mechanism in the Z-axis component, realizing the focusing function of the industrial camera;

[0009] The thin pad has the characteristics of being flat and wrinkle-free, opaque, having strong light reflection ability, and being non-mirror smooth;The glass substrate is placed on the thin pad, and the thin pad is placed on the auxiliary platform;The coarse positioning fixture positions the glass substrate within the tolerance;

[0010] A glass substrate micropore detection method, characterized in that, including: device preparation stage, platform motion trajectory planning, micropore missed marking and rechecking;

[0011] Preferably, the device preparation stage includes the following steps:

[0012] S1: the coarse positioning fixture is opened, and the glass substrate to be detected is placed on the thin pad, and the coarse positioning fixture positions the glass substrate within the tolerance;

[0013] S2: according to the size of the micropore, set the magnification of the with light source variable power lens, set the parameters of the industrial camera, and rotate the adjusting handle to obtain a clear micropore image with uniform illumination;

[0014] Preferably, the platform motion trajectory planning includes the following steps:

[0015] S3: the coarse positioning fixture positions the glass substrate within the tolerance and each photo has an overlapping area with the adjacent photo, so that the effective rectangular image size of the first row and the first column of photos is greater than the center interval of the adjacent photos in the same row, and the width is greater than the center interval of the adjacent photos in the same column;According to the field size determined in the step S2 and the size of the glass substrate, it is calculated that a rows of b columns of photos are needed to collect the image of the whole glass substrate;

[0016] S4: the industrial camera moves in an "S" trajectory, fixed row and column interval to capture photos; compare the number of corners in the "row" and "column" respectively as the starting to collect the complete block glass substrate, the scheme with fewer corner numbers as the industrial camera and the auxiliary platform motion trajectory;

[0017] S5: Mark point image acquisition: establish a world coordinate system for the industrial camera and the auxiliary platform and its motion origin, record the position of the industrial camera in the world coordinate system when capturing Mark point images and extract three Mark point pixel coordinates;

[0018] S6: rotation angle calculation: the actual position of the glass substrate on the auxiliary platform is considered to be obtained by translation and rotation of the DXF template file position, a coordinate system is established for the micro-hole processing DXF template file of the glass substrate, the DXF template file is parsed to obtain the coordinates and radius of the micro-hole, which are stored in the program matrix mat, and the rotation angle of the glass substrate is calculated:

[0019] S7: motion trajectory and method planning of image acquisition: obtain the first row and first column image, then, when the row / column direction is the starting motion:

[0020] Eg1: when the center interval E / D between adjacent photos in the same row / column is large, in the row / column direction, the industrial camera and the auxiliary platform move towards each other at a planned speed and trajectory at non-photographing positions; at photographing positions, one of the industrial camera and the auxiliary platform moves at a low uniform speed in a straight line, the other reaches the photographing position and stops moving, or both move towards each other at a low uniform speed to collect images;

[0021] Eg2: when the center interval E / D between adjacent photos in the same row / column is small, in the row / column direction, the industrial camera and the auxiliary platform move towards each other at a uniform speed in a straight line throughout the process (except at trajectory corners), and move towards each other at a planned trajectory to collect images at corresponding positions;

[0022] At the trajectory corner, the industrial camera and the auxiliary platform move towards each other, stop moving at the target position, and then collect images;

[0023] The industrial camera and the auxiliary platform move towards each other, and between two adjacent image acquisition positions, they move together in the column direction by D=ΔY+ΔY1, or in the row direction by E=ΔX+ΔX1, where D and E are fixed values, and ΔY, ΔY1, ΔX and ΔX1 are random values, which depend on real-time cooperative motion and are planned according to the principle of time optimization; when photographing, the absolute coordinates of the industrial camera and the auxiliary platform are X, Y and X1, Y1 respectively, and

[0024]

[0025] wherein int is an integer function, round is a rounding function, M is the image label collected at this position and stored with X, Y, X1, Y1 values and image.

[0026] As preferred, the micro-hole miss-marking and re-inspection comprises the following steps:

[0027] S8: image processing and coordinate transformation:

[0028] S81: the image is subjected to Gaussian filtering, binarization, and then all connected domains S and their parameters are obtained by connectedComponentsWithStats function, and according to the area s min ≤s i ≤s max The connected domain S meeting the actual micro-hole is screened out i ; the center of mass of each micro-hole connected domain is calculated According to the z value, the rotation direction of the glass substrate and the positive and negative of the corresponding sinθ value are determined.

[0029] S82: the S i is transformed from the pixel coordinate system μo0υ to the xoy coordinate system, and (x` i ,y` i ) is obtained; then the center of mass (x` i ,y` i ) of each micro-hole connected domain is searched for a matching coordinate point (x i ,y i ) in the matrix mat, which satisfies the constraints |x` k -x|<δ and |y` k -y|<δ at the same time, (x` i ,y` i ) is matched to a coordinate point (x k ,y k ), (x k ,y k ) is stored in the matrix mat1 in the program, and (x i ,y i ) continues to be matched with the remaining coordinates of the matrix mat; in order to make the matching point of the mass point (x` i ,y` i ) in the matrix mat not more than one and paired with the nearest coordinate point, δ needs to satisfy δ<0.5L min , L min is the shortest distance between any two circle centers in the DXF template file; after all images are collected, processed, and coordinate-transformed and matched in the above manner, the miss-marking coordinates (x j ,yj );

[0030] S9:Micro-hole missed mark and recheck:

[0031] S91:(x j ,y j ) into the following formula

[0032]

[0033] Find the same number "M" as "m", get the missed point (x j ,y j ) in the image with label "M", where trunc is the integer function;

[0034] S92:In the image with label "M", the xoy coordinate system point (x j ,y j ) is first transformed by inverse translation rotation, and then transformed by coordinate system transformation into the pixel coordinate system μo0υ, the pixel coordinate (μ j ,υ j ) in the image can be calculated, and then the pixel coordinate point is taken as the center and R is taken as the radius to draw a circle, the missed micro-hole position is marked on the image and displayed on the display screen;

[0035] S10:Develop a recheck system, when the missed micro-hole position is displayed on the screen, the system prompts "confirm whether the micro-hole is missed", after confirmation by manual or other judgment method, reply "yes", then rename the image with "m", and store the renamed and marked image with (x j ,y j ); reply "no", then do not store (x j ,y j ) and the image; the recheck system can be shielded, after shielding, the system no longer prompts "confirm whether the micro-hole is missed", the program directly enters the image with "m" renaming, and stores the renamed and marked image with (x j ,y j ). After the glass substrate micro-hole detection is completed, only the missed micro-hole coordinates (x j ,y j ) and the marked image are saved, both of which are stored in bundles, and other detection process data is deleted. If there are multiple micro-holes in an image, they should be uniformly marked in the image.

[0036] The computer system comprises at least one processor, at least one memory, and computer program instructions stored in the memory, which realize the above-mentioned method when executed by the processor.

[0037] A glass substrate micropore detection storage medium, characterized in that a storage medium is provided with computer program instructions, and the computer program instructions are executed by a processor to implement the method.

[0038] From the above technical solutions, the application has the following advantages: the application discloses a double-platform cooperative high-speed micropore detection system which can be quickly transplanted, has the advantages of adapting to fast product updates, visualizing detection results, and low cost, and the like. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 A glass substrate micropore detection device schematic diagram is provided for the embodiments of the application.

[0040] Figure 2 A glass substrate micropore detection image acquisition and platform motion trajectory planning method schematic diagram is provided for the embodiments of the application.

[0041] Figure 3 A glass substrate micropore processing DXF template file schematic diagram is provided for the embodiments of the application.

[0042] Figure 4 A glass substrate position translation and rotation schematic diagram is provided for the embodiments of the application.

[0043] Figure 5 A glass substrate micropore detection positioning principle diagram is provided for the embodiments of the application.

[0044] Figure 6 A glass substrate micropore detection method flowchart is provided for the embodiments of the application.

[0045] Among them:

[0046] 1Gantry motion platform 2Auxiliary platform

[0047] 3Expansion mirror 4Variable magnification mirror with light source

[0048] 5Industrial camera 6Coarse positioning clamp

[0049] 7Lens holder 8Lens holder

[0050] 9Z-axis assembly 10Adjusting handle

[0051] 11Computer system 12X-axis

[0052] 13Thin pad 14Glass substrate

[0053] 15Y-axis HGlass substrate length

[0054] L glass substrate width DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. The drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor under the premise of not paying any creative labor.

[0056] The glass substrate TGV (through-glass vias) technology uses laser to punch tens of thousands of micro-holes on the glass substrate. Due to the instability of the laser equipment and other reasons, some micro-holes are missed on the glass substrate. During production, we need to check out the missed micro-holes and obtain their coordinate positions, and then use laser to re-punch the missed micro-holes to ensure the product in this process. Generally, we will use the method of deep learning to detect the missed holes, but when using the deep learning algorithm, a large amount of data samples and computing resources are needed for long-time training of the model, and the model cannot be quickly put into production after simple adjustment of parameters to adapt to the current situation of fast updating of glass substrate product models.

[0057] The method, device and storage medium for detecting micro-holes of a glass substrate provided by the present application use a coordinate point matching method to find missed holes. Compared with the image template matching method based on the deep learning algorithm, less computing resources are needed. When the size of the glass substrate, the size of the micro-holes, the spacing between the micro-holes, the arrangement of the micro-holes and other parameters change, only the values corresponding to the parameters in the program need to be changed, so that the detection system can quickly adapt to the production of new products, and has the characteristics of strong compatibility. Unlike the deep learning algorithm, which needs to collect a large number of image samples to train the model for a long time before it can be put into production of new products. In view of the problem that it is difficult to obtain high motion speed in a short stroke of the motion platform, the method of moving the two platforms towards each other is used to improve the image acquisition efficiency. In summary, the method, device and storage medium for detecting micro-holes of a glass substrate provided by the present application can not rely on a large number of data samples and long-time training of the model, and provides a set of double-platform cooperative high-speed micro-hole detection system which can be quickly transplanted by using conventional computing resources, and has obvious low-cost advantage.

[0058] As shown in Figure 1 Fig. 1, a glass substrate micro-hole detection device mainly includes a gantry motion platform 1, an auxiliary platform 2, a magnifying lens 3, a variable magnification lens with a light source 4, an industrial camera 5, a coarse positioning clamp 6, a lens holder 7, a lens holder 8, a Z-axis assembly 9, an adjusting handle 10, a computer system 11, etc. The gantry motion platform 1 and the auxiliary platform 2 can realize planar motion and the motion planes of the two are parallel, so as to ensure the imaging quality of the industrial camera.

[0059] The magnification of the zoom lens 4 with light source is adjustable within a certain range, which is used for compatible imaging of micro-holes with different sizes; the magnification of the expansion lens 3 is fixed, which mainly serves to increase the magnification of the optical magnification system; the optical magnification system is composed of the expansion lens 3, the zoom lens 4 with light source, and the lens barrel, etc., to realize the multi-stage optical magnification function; the optical magnification system is fixed on the Z-axis assembly 9 through the lens holder 7 and the lens holder 8; the industrial camera 5 is connected with the lens of the expansion lens 3.

[0060] The Z-axis assembly 9 is fixed on the X-axis 12 mover of the gantry motion platform 1, to realize the synchronous motion of the X-axis 12 mover of the gantry motion platform 1 and the Z-axis assembly 9. The adjusting handle 10 moves the optical magnification system in the Z-axis direction through the slide-screw mechanism in the Z-axis assembly 9, to realize the focusing function of the industrial camera 5.

[0061] The thin pad 13 has the characteristics of flatness, no wrinkles, opacity, strong light reflection ability, and non-mirror smoothness, to realize the formation of dark and bright areas on the micro-holes and the non-punched surface of the glass substrate 14. The glass substrate 14 is placed on the thin pad 13, the thin pad 13 is placed on the auxiliary platform 2, and the coarse positioning clamp 6 is used for positioning the glass substrate 14 within the tolerance; the positioning reduces the rotation angle of the glass substrate 14, thereby reducing the actual area of the glass substrate 14 that needs to be collected by the industrial camera 5, and also lays the foundation for the positioning of the missed holes; the positioning within the tolerance prevents the glass substrate 14 from being crushed.

[0062] The glass substrate micro-hole detection method includes three steps: device preparation stage, platform motion trajectory planning, and micro-hole missed marking and re-inspection.

[0063] The device preparation stage includes the following steps:

[0064] S1: The coarse positioning clamp 6 is opened, and the glass substrate 14 to be detected is placed on the thin pad 13, and the coarse positioning clamp 6 is used for positioning the glass substrate 14 within the tolerance;

[0065] S2: The magnification of the zoom lens 4 with light source is set according to the size of the micro-hole, the parameters of the industrial camera 5 are set, the adjusting handle 10 is rotated, a clear micro-hole image with uniform illumination is obtained, the industrial camera 5 needs to be calibrated at first use or product change, and the pixel scale K is obtained.

[0066] The platform motion trajectory planning includes the following steps:

[0067] S3: As Figure 2As shown, the coarse positioning fixture 6 positions the glass substrate 14 with positive tolerance, and each photograph overlaps with its adjacent photograph by a region ΔH*ΔL. This ensures that the effective rectangular image size of the first row and first column photograph has a length F slightly larger than the center-to-center distance E between adjacent photographs in the same row, and a width G slightly larger than the center-to-center distance D between adjacent photographs in the same column. Based on the field of view determined by the industrial camera 5 in step S2 and the size H*L of the glass substrate 14, it is calculated that a rows and b columns of photographs are needed to capture an image of the entire glass substrate 14.

[0068] S4: The industrial camera 5 moves in an "S" shaped trajectory and captures photos at fixed row and column intervals; the number of corners encountered from the start of the acquisition of the entire glass substrate 14 is compared with the number of corners encountered by the industrial camera 5 and the auxiliary platform 2.

[0069] S5: Mark Point Image Acquisition: Establish the following for industrial camera 5 and auxiliary platform 2 respectively: Figure 1 The world coordinate systems shown are O-XYZ and O-X1Y1Z1. (Example) Figure 2 and Figure 3 As shown, A and P0, B and P m These are the same point in different coordinate systems. The image acquisition position of the first group of Mark points is the first row and first column of the first photo acquisition position. In this case, this position is also the image acquisition starting point O0. At this time, the gantry motion platform 1, auxiliary platform 2, and industrial camera 5 are at the origin. The extracted pixel coordinates of the Mark points are... The second set of Mark point image acquisition positions is the image acquisition position in row a and column b. In this case, this position is also the image acquisition endpoint O. m Extract the pixel coordinates of the Mark point as Third group of Mark points P n As a backup, in case the two sets of Mark points cannot uniquely determine the state of glass substrate 14. Industrial camera 5 moves from O0 to O. m It moved X in the positive X-axis direction. m It moved Y in the positive Y-axis direction m Record the position of the Mark point in the world coordinate system when the industrial camera 5 acquires the Mark point image.

[0070] S6: Rotation angle calculation: such as Figure 4 As shown, the actual position of the glass substrate 14 on the auxiliary platform 2 is considered to be obtained by translating and rotating the position of the DXF template file. The DXF template file for micro-hole processing of the glass substrate 14 is established as follows: Figure 3 The xoy coordinate system shown is used to parse the DXF template file, obtaining the coordinates (x, y) and radius r of the micro-hole, which are stored in matrix mat in the program. On the xoy plane, A = (0, 0), B = (x...y...r)... B y B ), The pixel coordinates P0, P m Transformed into the xoy plane coordinate system, we have: where K is the pixel scale (mm / pixel). The glass substrate rotation angle is calculated as:

[0071] The micro-hole size is small and needs to be optically magnified to be processed by the visual algorithm. However, the field of view of the industrial camera 5 is small, and usually hundreds or thousands of photos are needed to collect the complete image of the glass substrate 14. In order to improve the efficiency of image acquisition, the commonly used method is to increase the motion speed of the platform, but the field of view of the industrial camera 5 is small, and it is difficult to obtain high motion speed in a short stroke. To solve this problem, the present application adopts two methods to improve the efficiency of image acquisition, one is to use the double-platform opposite motion mode as shown in the figure, and the other is to use the mode that at least one of the moving platforms does not stop moving when taking photos. Figure 2

[0072] S7: Motion trajectory and mode planning for image acquisition: first, collect the image located in the first row and the first column, and when step S4 determines that the industrial camera 5 starts to move with the column as the starting point, as shown in the figure: Figure 2

[0073] Eg1: When the center interval D between adjacent photos in the same column is large: within the center interval D, the industrial camera 5 and the auxiliary platform 2 can obtain a motion speed, which is greater than the maximum speed v max of the industrial camera 5 under the condition of uniform linear motion without producing trailing when taking photos. At this time, in the column direction, the industrial camera 5 and the auxiliary platform 2 move in the planned speed and trajectory in the opposite direction; at the photo position, one of the industrial camera 5 and the auxiliary platform 2 moves at a low uniform linear speed (speed is 0.5v max ), and the other stops moving, or both move in the opposite direction at a low uniform speed (speed is 0.5v max ) to collect images.

[0074] Eg2: When the center interval D between adjacent photos in the same column is small: the motion speed obtained by the industrial camera 5 and the auxiliary platform 2 is less than the maximum speed v max of the industrial camera 5 under the condition of uniform linear motion without producing trailing when taking photos. At this time, in the column direction, the industrial camera 5 and the auxiliary platform 2 move in the opposite direction at a uniform linear speed (speed is 0.5v max ) throughout the trajectory (except at the corners of the trajectory) and move in the planned trajectory to the corresponding position to collect images.

[0075] When step S4 determines that the industrial camera 5 starts to move with the row as the starting point, similarly, the motion trajectory and mode of image acquisition are planned in the above Eg1 and Eg2 two cases.​​

[0076] At the corner of the trajectory, the industrial camera 5 and the auxiliary platform 2 adopt a coordinated movement mode, reach the target position and stop moving, and then image acquisition is performed.

[0077] The industrial camera 5 and the auxiliary platform 2 move together, and at the adjacent two image acquisition positions, the industrial camera 5 and the auxiliary platform 2 move together by D = ΔY + ΔY1 in the column direction or E = ΔX + ΔX1 in the row direction, wherein D and E are fixed values planned in step S4, and ΔY, ΔX, ΔY1 and ΔX1 are random values, which represent the moving amount of the industrial camera 5 in the Y-axis 15 and X-axis 12 directions and the moving amount of the auxiliary platform 2 in the Y1-axis and X1-axis directions at the adjacent two image acquisition positions, and are planned according to the time optimization principle of real-time coordinated movement; when photographing, the absolute coordinates of the industrial camera 5 in the world coordinate system O-XYZ are X, Y, and the absolute coordinates of the auxiliary platform 2 in the world coordinate system O-X1Y1Z1 are X1 and Y1, and the following calculation is performed:

[0078]

[0079] Wherein int is the integer function, round is the rounding function, M is the image label collected at this position and is stored in bundle with X, Y, X1 and Y1.

[0080] Wherein, the micro-hole missed marking and re-inspection includes the following steps:

[0081] S8: Image processing and coordinate transformation:

[0082] S81: After the image is subjected to Gaussian filtering and binarization, all connected domains S and their parameters are obtained by the connectedComponentsWithStats function, and according to the area s min ≤s i ≤s max The connected domain S that meets the actual micro-hole is screened out i ; the calculation Figure 1 O-XYZ is the left-handed coordinate system, if z > 0, the glass substrate is judged to rotate clockwise according to the left-hand rule, if z < 0, the glass substrate is judged to rotate counterclockwise according to the left-hand rule,

[0083] S82: S i is transformed into the xoy coordinate system, and (x` i ,y` i ) is obtained; then the centroid (x` i ,y` i) in the matrix mat that satisfy the constraints |x i -x| < δ and |y i -y| < δ simultaneously, find the coordinate point (x k ,y k ) that matches them, (x i ,y i ) matches the coordinate point (x k ,y k ), store (x k ,y k ) in the matrix mat1 in the program, and immediately stop (x i ,y i ) from continuing to match the remaining coordinates in the matrix mat; in order to make the mass point (x i ,y i ) find no more than one matching point in the matrix mat and pair with the nearest coordinate point, δ needs to satisfy δ < 0.5L min , L min is the shortest distance between any two circle centers in the DXF template file. After all images are collected, processed, and coordinate-transformed and matched in the above manner, the hole center coordinates (x j ,y j ) can be obtained by comparing the matrix mat and the matrix mat1.

[0084] The hole is found in step S82 using the coordinate point matching method, and compared with the traditional deep learning algorithm based on image template matching method, the computing resources required will be less.

[0085] Because the placement position of each glass substrate 14 on the thin pad 13 is different, that is, it is not possible to achieve that each image corresponds to a small area DXF template file, therefore, in step S82, after all images are collected, processed, and coordinate-transformed and matched in the above manner, the hole center coordinates (x j ,y j ) can be obtained by comparing the matrix mat and the matrix mat1. In order to reduce the demand for computing resources, the mass point coordinates (x i ,y i ) corresponding to the connected domain Si are not stored for the hole center coordinates (x j ,y j ) to match, so as to be located in the original image, and finally identify the hole. In view of this problem, the following method with less demand for computing resources is adopted, which uses the following two conditions:

[0086] (a) The industrial camera 5 and the auxiliary platform 2 jointly move at a fixed interval D and E each time the image is collected.

[0087] (b) Positioning the glass substrate 14 to the positive tolerance such that the first row first column photo active rectangular image size F*G, length F is slightly larger than the adjacent photo center spacing E in the same row, width G is slightly larger than the adjacent photo center spacing D in the same column. Each photo has an overlapping area with the adjacent photo.

[0088] Each photo has an overlapping area with the adjacent photo, one of the purposes is to prevent some micro-hole image part in one photo and another part in another photo at the photo junction, which is connected domain area s min ≤s i ≤s max is filtered out as a false positive. Another purpose is to construct condition (b).

[0089] As Figure 5 shown, the above two prerequisites make the micro-hole center coordinate point (x n ,y n ) in DXF template file when 0≤x n ≤E, x n falls in the range of F, that is, the first column photo range; when E≤x n ≤2E, x n falls in the range of F1, that is, the second column photo range; when 2E≤x n ≤3E, x n falls in the range of F2, that is, the third column photo range; similarly, when 0≤y n ≤D, y n falls in the range of G, that is, the first row photo range; when D≤y n ≤2D, y n falls in the range of G1, that is, the second column photo range; when 2D≤y n ≤3D, y n falls in the range of G2, that is, the third column photo range; other ranges x n , y n also have the above rules, where F1=F2=F3=…, G1=G2=G3=… Therefore, we use the rule to calculate the x j / E, y j / D ratio under the condition of knowing the micro-hole coordinate point (x j ,y j ), which can know the micro-hole connected domain center of mass (x` j ,y` j ) in which row and column photo. Similarly, for the world coordinate system O-XYZ and the world coordinate system O-X1Y1Z1 coordinate (X n ,Y n ) also have the same above rules, where Xn = |X1| + |X|,Y n = |Y1| + |Y| The specific method is as follows:

[0090] S9: Micro-hole missed marking and rechecking:

[0091] S91: (x j ,y j ) is substituted into the following formula

[0092]

[0093] The same number "M" as "m" is found, and the missed point (x j ,y j ) is obtained in the image with the label "m", wherein trunc is the integer function.

[0094] In the above, formula (1) uses the coordinate positions of the industrial camera 5 and the auxiliary platform 2 when collecting images to give the collected images four-digit encoding. The ① part in formula (1) is to obtain the first two digits to represent the column of the photo, and the ② part is to obtain the last two digits to represent the row of the photo. Because there is ambiguity in interpreting the meaning of three or fewer digits, for example, the code "921" can be translated into "09, 21" and "92, 01" with four-digit numbers, corresponding to two images. The "+10" in formula (1) and formula (2) is to make the code start from "1010", at which time one four-digit number uniquely corresponds to one image. If the number of photos is large, more digits can be considered for encoding.

[0095] S92: In the image with the label "M", the xoy coordinate system point (x j ,y j ) is first transformed through inverse translation rotation, and then transformed through coordinate system transformation into the pixel coordinate system μo0υ, and the pixel coordinate (μ j ,υ j ) in the image can be calculated. Then, taking the pixel coordinate point as the center and R as the radius, a circle is drawn to mark the missed micro-hole position on the image and display it on the display screen.

[0096] In formula (2), the xoy coordinate system point (x j ,y j ) is transformed through inverse translation rotation to obtain the corresponding (x` j ,y` j ), and (x` j ,y` j ) is positioned in the image with the same label "m" through the calculation of ③ and ⑤.

[0097] S10: a set of rechecking system is developed to artificially recheck or spot check the correctness of the detection result of the glass substrate detection system. When the missed micro-hole position is displayed on the screen, the system prompts "confirm whether the micro-hole is missed". After artificial or other judgment methods are confirmed, the answer is "yes", the image is renamed as "m", and the renamed and marked image and (x j ,y j ) are bundled and stored; the answer is "no", and (x j ,y j ) and the image are not stored; the rechecking system can be shielded, and after shielding, the system no longer prompts "confirm whether the micro-hole is missed", and the program directly enters the image renamed as "m", and the renamed and marked image and (x j ,y j ) are bundled and stored.

[0098] In order to save the storage resources of the computer, only the missed micro-hole coordinates and the marked image after the detection of the missed micro-hole of the glass substrate 14 are saved, and the two are bundled and stored, and other detection process data is deleted. If there are multiple missed holes in an image, they should be uniformly marked in this image. The flow method of the glass substrate micro-hole detection is shown in Figure 6 .

[0099] The computer system 11 includes at least one processor, at least one memory, and computer program instructions stored in the memory, which realize the above-mentioned method when the computer program instructions are executed by the processor.

[0100] A glass substrate micro-hole detection storage medium, a storage medium having computer program instructions stored thereon, which realize the above-mentioned method when the computer program instructions are executed by the processor.

[0101] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above-mentioned disclosed technical content without departing from the scope of the technical solution of the present application, and any equivalent embodiments with equivalent changes are equivalent to the above-mentioned embodiments. Any modification, change, equivalent change and modification of the above-mentioned embodiments according to the technical essence of the present application are still within the scope of the technical solution of the present application.

Claims

1. A glass substrate microvoid detection apparatus, characterized by, It comprises: Gantry motion platform, auxiliary platform, expansion mirror, variable magnification mirror with light source, industrial camera, coarse positioning clamp, two sets of lens holder, Z-axis assembly, adjusting handle, computer system; the gantry motion platform and the auxiliary platform can realize planar motion and the motion planes of the two are parallel; The variable magnification mirror with light source has adjustable magnification in a certain range, and the expansion mirror has fixed magnification; the expansion mirror, the variable magnification mirror with light source, and a lens barrel constitute an optical magnification system; the optical magnification system is fixed on the Z-axis assembly through the two sets of lens holders; the industrial camera is connected with the expansion mirror lens; The Z-axis assembly is fixed on the X-axis mover of the gantry motion platform, and the adjusting handle moves the optical magnification system in the Z-axis direction through the slide-screw mechanism in the Z-axis assembly, thereby realizing the focusing function of the industrial camera; The thin pad has the characteristics of flatness, no wrinkles, opacity, strong light reflection ability, and non-mirror smoothness; the glass substrate is placed on the thin pad, and the thin pad is placed on the auxiliary platform; the coarse positioning clamp performs positive tolerance positioning on the glass substrate.

2. The glass substrate microvoid detection apparatus of claim 1, wherein The glass substrate micro-hole detection method comprises a device preparation stage, a platform motion trajectory planning, a micro-hole missed marking identification, and a recheck. The device preparation stage comprises the following steps: S1: the coarse positioning clamp is opened, the glass substrate to be detected is placed on the thin pad, and the coarse positioning clamp performs positive tolerance positioning on the glass substrate; S2: according to the size of the micro-hole, the magnification of the variable magnification mirror with light source is set, the parameters of the industrial camera are set, and the adjusting handle is rotated to obtain a clear micro-hole image with uniform illumination; The platform motion trajectory planning comprises the following steps: S3: the coarse positioning clamp performs positive tolerance positioning on the glass substrate, and each photo has an overlapping area with the adjacent photo, so that the effective rectangular image size of the first row and the first column of photos is slightly larger than the center interval of the adjacent photos in the same row, and the width is slightly larger than the center interval of the adjacent photos in the same column; according to the field of view size determined in step S2 and the size of the glass substrate, the number of rows and columns of photos required to collect the image of the whole glass substrate is calculated; S4: the industrial camera moves in an "S" trajectory and collects photos with fixed row and column intervals; compare the number of corners experienced by the industrial camera and the auxiliary platform when starting from "row" and "column" to collect the complete glass substrate, and the scheme with fewer corners is selected as the motion trajectory of the industrial camera and the auxiliary platform; S5: Mark point image collection: a world coordinate system and its motion origin are established for the industrial camera and the auxiliary platform, the position of the industrial camera in the world coordinate system when collecting the Mark point image is recorded, and three Mark point pixel coordinates are extracted; S6: rotation angle calculation: the actual position of the glass substrate on the auxiliary platform is considered to be obtained by the position translation and rotation of the DXF template file, a coordinate system is established in the DXF template file for the micro-hole processing of the glass substrate, the DXF template file is analyzed to obtain the center coordinates and the radius of the micro-hole, which are stored in the matrix mat in the program, and the rotation angle of the glass substrate is calculated: S7: motion trajectory and mode planning of image collection: the first row and the first column of images are obtained, and then, when starting from the row / column direction: Eg1: When the center distance E / D between adjacent photos in the same row / column is large, in the row / column direction, the industrial camera and the auxiliary platform move towards each other at a planned speed and trajectory in a non-photographing position; at a photographing position, one of the industrial camera and the auxiliary platform moves at a low constant speed in a straight line, the other reaches the photographing position and stops moving, or both move towards each other at a low constant speed to collect images; Eg2: When the center distance E / D between adjacent photos in the same row / column is small, in the row / column direction, the industrial camera and the auxiliary platform move towards each other at a constant speed in a straight line throughout the trajectory (except at the trajectory corner), and move towards each other at a planned trajectory to collect images at the corresponding position; At the trajectory corner, the industrial camera and the auxiliary platform move towards each other, stop moving at the target position, and then collect images; The industrial camera and the auxiliary platform move towards each other, and move D = ΔY + ΔY1 in the column direction or E = ΔX + ΔX1 in the row direction between two adjacent image collection positions, where D and E are fixed values, and ΔY, ΔY1, ΔX and ΔX1 are random values, which depend on the real-time coordinated motion and are planned according to the principle of time optimization; when photographing, the absolute coordinates of the industrial camera and the auxiliary platform are calculated as X, Y and X1, Y1 respectively Where int is the integer function, round is the rounding function, M is the image label collected at this position and is stored in bundle with the values of X, Y, X1 and Y1. The micro-hole missed marking identification and re-inspection includes the following steps: S8: Image processing and coordinate transformation: S81: image is filtered by Gauss, binarized and then all connected domains S and their parameters are obtained by connectedComponentsWithStats function, according to area s min ≤s i ≤s max Connected domains S meeting the actual micropore are screened out i ; Calculate According to the value of z, the rotation direction of the glass substrate and the positive and negative of the corresponding sinθ value are judged. S82: put S i The corresponding each micro-hole communication area centroid coordinate is transformed from the pixel coordinate system μo0υ to the xoy coordinate system, and (x` i , y` i ,) is obtained; then each micro-hole communication area centroid (x` i , y` i ,) is stored in the matrix mat, and the coordinate point (x i , y i ) matched therewith is searched according to the constraints |x` k -x| < δ and |y` k -y| < δ; (x` i , y` i ,) is matched to a coordinate point (x k , y k ), (x k , y k ) is stored in the matrix mat1 in the program, and immediately stops (x` i , y` i ,) and the remaining coordinates of the matrix mat continue to be matched; δ needs to satisfy δ < 0.5L min , L min is the shortest distance between any two circle centers in the DXF template file; after all images are collected, processed, and coordinate-transformed and matched in the above manner, the comparison between the matrix mat and the matrix mat1 can obtain the missing point coordinates (x j , y j ). S9: Micro-hole missed marking identification and re-inspection: S91: (x j ,y j ) into the following equation Find the same number "M" as "m" from the leak point (x j ,y j ) in the image labeled "M", where trunc is the truncation function; S92: In the image with label "M", the point (x j ,y j ) is first transformed by inverse translation rotation, and then by coordinate system transformation, to the pixel coordinate system μo0υ. The pixel coordinate (μ j ,υ j ) in the image can be calculated. Then, a circle is drawn with the pixel coordinate point as the center and R as the radius, the missed micro-hole position is marked on the image and displayed on the display screen. S10:Develop a review system, when the missed micro-hole position is displayed on the screen, the system prompts "confirm whether the micro-hole is missed", after the confirmation of manual or other judgment methods, the answer is "yes", then rename the image as "m", and store the renamed and marked image with (x j ,y j ) in bundle; the answer is "no", then do not store (x j ,y j ) and the image; the review system can be shielded, after the shielding, the system no longer prompts "confirm whether the micro-hole is missed", the program directly enters the image renamed as "m", and stores the renamed and marked image with (x j ,y j ) in bundle; after the detection of the missed micro-hole of the glass substrate, only save the missed micro-hole coordinates (x j ,y j ) and the marked image, both of which are stored in bundle, and other detection process data is deleted; if there are multiple missed holes in an image, they should be uniformly marked in this image.

3. The glass substrate micro-hole detection device according to claim 1, wherein The computer system includes at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of any one of claims 2.

4. A glass substrate microvoid detection storage medium characterized by, A storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the method of any one of claims 2 is implemented.

Images