Glass edge defect detection and pose recognition method and system

Abstract

The invention discloses a glass edge defect detection and pose recognition method and system, and relates to the technical field of glass edge detection, and the method comprises the following steps: S1, collecting and preprocessing an original image of to-be-detected glass to obtain a preprocessed image; s2, performing edge extraction on the preprocessed image based on direction multi-resolution gray morphology to obtain a to-be-detected contour image of the to-be-detected glass; s3, performing primary registration on the to-be-detected contour image and the template contour image based on a multi-resolution layered progressive image search strategy, outputting an identification result and acquiring pose information of the to-be-detected glass; and S4, performing secondary registration on the to-be-detected contour image and the template contour image based on contour arch height vector similarity, and positioning and identifying defects in the to-be-detected contour image. According to the method, the glass edge defects can be detected with high real-time performance and high precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of glass edge detection, in particular to a glass edge defect detection and pose recognition method and system. BACKGROUND

[0002] At present, float glass is widely used in the fields of building, household appliances and automobile due to its good optical performance and small internal stress. Surface defects of glass occur in the hot end and cold end of the float glass production process. The surface defects that are prone to occur in the hot end include bubbles, inclusions, light distortion spots, tin sticking, lines and the like. These surface defects can be detected by a special defect detector. The cold end defects are mostly located at the edge and the surrounding parts of the glass, such as corner fracture, notch, burr, edge collapse, edge explosion, crack and edge pulling indentation. If the defects exceed the limit set by the user, the overall quality of the glass will be affected, and the glass exceeding the limit needs to be disposed of in time. Therefore, the detection of glass edge defects is particularly important.

[0003] The glass edge detection methods mainly include manual detection, laser detection, Moire interference method detection, grating modulation method detection and machine vision method detection. The manual detection mainly relies on the judgment of the glass shape, size and defects by the operator. Experiments show that the glass defects that can be accurately detected by the human eye are usually greater than 0.5 mm and have large optical deformation. Therefore, this detection method has low precision, strong subjectivity and high omission rate of 15-20%. At the same time, the human eye cannot realize online defect detection in high-speed production, and cannot meet the requirements of online real-time monitoring of the whole process of production quality.

[0004] The laser detection utilizes special laser display to effectively amplify the defects before the electrical signal conversion. However, the structure of this detection method is complex, and it is difficult to popularize and apply.

[0005] The Moire interference method detection is performed by Moire fringe pattern generated by light interference. The Moire fringes generated by light transmission through the glass will be obviously deformed at the defect position, and the sensitivity is high. However, this detection method needs the support of high hardware system, and is easily disturbed by dust and vibration, which makes the on-site debugging difficult. For example, the Floatsean-Cateher & Optics automatic detection equipment developed by the German Innomess company has the principle of Moire fringe interference. The identification and classification of defects are realized by relying on the different calculation results of the Moire detection probe on the light distortion at the defect position. Although the detection range of this equipment is very comprehensive, and the application range is also relatively wide, the grating line precision is directly related to the system detection precision, which leads to high system cost and high maintenance cost in the later period.

[0006] The grating modulation method detection is that a CCD (Charge Coupled Device) camera is placed on one side of the glass plate to collect a grating fringe image on the other side of the glass plate, and the grating fringe image is formed by the cooperation of the light source 4 and the physical grating and then transmits through the glass, the method can detect various glass defects, and has high sensitivity, but the collected fringe image has uneven light intensity distribution, so that the contrast between the target and the background is not high.

[0007] The machine vision method detection utilizes a camera to collect glass images on a production line in real time, cooperates with a high-speed processor to synchronously process a large amount of data, changes the gray value of the image according to the influence of the glass defect on the optical fiber, identifies the image and classifies the defects, the method has good real-time performance and high detection accuracy. However, in order to ensure the accuracy of the detection result, the calculation amount of the method is very large, and then the image processing time is relatively long, so that real-time detection cannot be realized.

[0008] The glass at the cold end is conveyed to the stacking device by the conveying device, the glass on the conveying device is grabbed by the mechanical arm of the stacking device and is subjected to stacking operation, however, in the process of conveying the glass, the pose of the glass changes, for example, angle rotation or position offset transverse misplacement, due to the vibration generated by the conveying roller 5 and the slight balance loss of bearing assembly, the above-mentioned pose change causes the glass in the stacking device to have stacking deviation, so that the glass is misaligned and stacked, and the electrostatic attraction between the glass is relatively large, which brings great trouble for subsequent deviation correction and packaging work. SUMMARY

[0009] In order to solve the problems of unable to realize real-time detection and misaligned and stacked glass in the prior art, the present application provides a glass edge defect detection and pose recognition method and system, which can detect glass edge defects with high real-time performance and high accuracy.

[0010] In order to achieve the above-mentioned purpose, the present application adopts the following specific scheme: a glass edge defect detection and pose recognition method, comprising the following steps:

[0011] S1, collecting an original image of a glass to be tested and pre-processing to obtain a pre-processed image;

[0012] S2, performing edge extraction on the pre-processed image based on direction-based multi-resolution gray morphology to obtain a to-be-tested contour image of the glass to be tested;

[0013] S3, performing one-time registration on the to-be-tested contour image and a template contour image based on a multi-resolution hierarchical progressive image search strategy, outputting a recognition result and obtaining pose information of the glass to be tested;

[0014] S4, performing secondary registration on the to-be-tested contour image and the template contour image based on contour arch height vector similarity, and positioning and identifying defects in the to-be-tested contour image.

[0015] As an optimization scheme of the above-mentioned glass edge defect detection and pose recognition method, in S1, the preprocessing of the original image comprises the following steps:

[0016] S101, filtering the original image by using Gaussian bilateral filtering to obtain a filtered image;

[0017] S102, performing threshold segmentation on the filtered image by using the maximum inter-class variance method Otsu to obtain a preprocessed image.

[0018] As another optimization scheme of the above-mentioned glass edge defect detection and pose recognition method, in S2, the method for obtaining the contour image comprises the following steps:

[0019] S201, constructing an edge model of the preprocessed image;

[0020] S202, performing edge extraction in 0 direction and π / 2 direction on the preprocessed image by using a one-dimensional morphological operator to obtain a longitudinal contour and a transverse contour;

[0021] S203, synthesizing the longitudinal contour and the transverse contour to obtain a to-be-detected contour image of the to-be-detected glass.

[0022] As another optimization scheme of the above-mentioned glass edge defect detection and pose recognition method, in S201, the edge model is:

[0023] f(x, y) = b + a(xcosθ + ysinθ);

[0024] Wherein, b is the background gray value of the preprocessed image, a is the contour amplitude value of the preprocessed image, and θ is the direction of the contour.

[0025] As another optimization scheme of the above-mentioned glass edge defect detection and pose recognition method, S3 comprises the following steps:

[0026] S301, layering the template contour image and the to-be-detected contour image according to the size of the resolution, and sequentially calculating the similarity measure between the corresponding layers of the template contour image and the to-be-detected contour image:

[0027]

[0028] Wherein, P(x, y) = G(f(x, y)) is the edge point set of the to-be-detected glass contour image, A(x, y) is the edge point set of the template glass image contour, and n is the number of to-be-detected glass contour points;

[0029] S302, comparing the calculated value of the similarity metric with a preset value. If the calculated value is less than the preset value, the match fails; if the calculated value is greater than or equal to the preset value, the match succeeds and the recognition result (x, y, θ) is output;

[0030] S303: Obtain the centroid coordinates of the glass to be tested and the rotation angle of the glass to be tested according to the recognition result (x, y, θ), and output them.

[0031] As another optimization solution of the above-mentioned glass edge defect detection and posture recognition method: S4 includes the following steps:

[0032] S401, dividing the contours in the contour image to be measured and the template contour image into equal parts to obtain multiple contour segments;

[0033] S402, calculating the amplitude of the height function of all points in each contour segment to obtain the height vector of each contour segment;

[0034] S403, calculating the similarity value between the height vectors of the contour segments corresponding to the contour image to be measured and the template contour image;

[0035] S405 , comparing the calculated value of the similarity value with a preset value. If the calculated value is greater than the preset value, a rectangle is used to mark the defect.

[0036] As another optimization solution of the above-mentioned glass edge defect detection and posture recognition method: in S402, the amplitude of the arch height function is:

[0037]

[0038]

[0039] A glass edge defect detection and posture recognition system includes a control device, a conveying device for conveying the glass to be tested, and an acquisition device for capturing images of the glass to be tested. The acquisition device includes a camera arranged above a conveying roller 5, so that an acquisition area is formed below the camera. The control device includes an image acquisition module, a signal acquisition module, a defect detection module, and a posture recognition module. The signal acquisition module is used to collect the conveying speed of the glass to be tested and control the camera's power supply. The image acquisition module is used to receive the original image.

[0040] As another optimization solution for the above-mentioned glass edge defect detection and posture recognition system: the conveying device includes a support frame and a plurality of conveying rollers rotatably arranged on the support frame, the plurality of conveying rollers are distributed along the conveying direction of the glass to be tested, and each conveying roller is provided with a rubber ring distributed along its axial direction.

[0041] As another optimization scheme of the above glass edge defect detection and pose recognition system, the interval between the two adjacent conveying rollers in the collection area is 150-200 mm, and the interval between the two adjacent rubber rings on the conveying roller in the collection area is 100-150 mm.

[0042] Compared with the prior art, the present application has the following beneficial effects: the present application provides a glass edge defect detection and pose recognition method, which can detect glass edge defects with high real-time performance and high precision, can recognize sub-millimeter level defects (such as 0.5 mm cracks) based on the to-be-tested contour image, and can judge whether the edge corners and edges have defects and whether to discard the glass according to the size of the defects, thereby improving the detection accuracy to more than 98%. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is each direction (from left to right, 0, π / 4, π / 2 and 3π / 4);

[0044] Figure 2 is the structure operator used in each direction (from left to right, 0 direction operator, π / 2 direction operator and 3×3 operator);

[0045] Figure 3 (a) is a preprocessed image, (b) is a longitudinal contour, (c) is a transverse contour, and (d) is a to-be-tested contour image;

[0046] Figure 4 is a multi-resolution hierarchical progressive image search strategy flowchart;

[0047] Figure 5 is an image ladder of a template contour image;

[0048] Figure 6 is an image ladder of a to-be-tested contour image;

[0049] Figure 7 is a secondary registration flowchart;

[0050] Figure 8 is a schematic diagram of an arch height vector;

[0051] Figure 9 is a schematic diagram of a rotatable minimum bounding rectangle;

[0052] Figure 10-Figure 20 is the detection result of various defects;

[0053] Figure 21 is a schematic diagram of a normal posture of intact glass;

[0054] Figure 22 is a schematic diagram of an inclined posture of intact glass;

[0055] Figure 23is a structural schematic diagram of the collecting device;

[0056] Figure 24 is a side view of the collecting device;

[0057] Figure 25 is a top view of the collecting device;

[0058] Figure 26 is a front view of the camera;

[0059] Figure 27 is a side view of the camera;

[0060] Figure 28 is a three-dimensional view of the camera;

[0061] Figure 29 is a structural schematic diagram of the second support and the light source;

[0062] Figure 30 is a top view of the second support and the light source;

[0063] Figures 31-32 is a schematic diagram of the second support;

[0064] Figure 33 is a top view of the conveying device;

[0065] Figure 34 is a side view of the conveying device;

[0066] Figure 35 is a front view of the conveying device;

[0067] Figure 36 is a schematic diagram of the encoder pulse conversion device;

[0068] Figure 37 is a schematic diagram of the photoelectric switch signal conversion device;

[0069] Figure 38 is a schematic diagram of the camera power supply interface;

[0070] Figure 39 is a schematic diagram of the DC / DC conversion;

[0071] Figure 40 is a display of the image and the processing result;

[0072] Figure 41 is a display of the image and the curve;

[0073] Figure 42 is a display of the image and the conveying speed;

[0074] Figure 43 is a hardware setting interface;

[0075] Figure 44is a flaw judgment rule setting interface;

[0076] Figure 45 is a camera debugging interface;

[0077] Figure 46 is a pattern position calibration;

[0078] Figure 47 is a pixel coordinate and physical coordinate calibration;

[0079] Figure 48 is a save, statistical analysis setting interface

[0080] Fig. 1 is a camera, 2, gantry, 3, the second support, 4, light source, 5, conveying roller, 6, shell, 7, mounting plate, 8, connecting plate, 9, shooting port, 10 spring, 11, knob. DETAILED DESCRIPTION

[0081] The technical solutions of the present application will be further described in detail below in combination with specific embodiments. The parts not described and disclosed in detail in the following embodiments of the present application should be understood as the prior art known or should be known by those skilled in the art.

[0082] Embodiment 1

[0083] A glass edge defect detection and pose recognition method, comprising the following steps:

[0084] S1, collecting the original image of the glass to be measured and pre-processing to obtain a pre-processed image; specifically, the pre-processing of the original image comprises the following steps:

[0085] S101, filtering the original image using Gaussian bilateral filtering to obtain a filtered image; filtering the original image using Gaussian bilateral filtering can preserve the edge linearity of the original image and suppress noise. Gaussian bilateral filtering combines the characteristics of Gaussian filtering and bilateral filtering, which not only reduces noise, but also preserves the edge and detail information of the image; the filter uses a Gaussian function to calculate the similarity between pixels, while considering spatial and grayscale distance, thereby realizing filtering processing of the original image. Specifically, the abscissa and ordinate directions of the glass contour points in the original image are smoothed, a 2D Gaussian smoothing operator G(x, y) is used to obtain a smoothed contour to suppress burr noise of the original image contour, and the 2D Gaussian smoothing operator is:

[0086]

[0087] Wherein, (x, y) represents the original image horizontal coordinate and vertical coordinate; σ standard deviation is the only parameter of the Gaussian filter, when constructing the Gaussian template, σ takes 1.6. The σ standard deviation is proportional to the size of the filter operation field, the farther away from the center of the Gaussian filter operator, the smaller the influence of the pixel on it.

[0088] S102, the maximum inter-class variance method Otsu is used for threshold segmentation of the filtered image, and a pretreated image is obtained; the threshold segmentation is realized by using the principle that the target and the background gray value are different, the gray value is converted from 0-255 to 0-1 through threshold segmentation, the calculation amount is greatly reduced, and irrelevant information such as noise is inhibited, the subsequent processing steps are simplified, and at the same time, the overall characteristics and local characteristics of the original image can still be reflected, and even the local contrast characteristics are more obviously embodied.

[0089] The maximum inter-class variance method Otsu (Otsu algorithm) is used, the original image can be divided into two regions of foreground and background, the variances of the foreground and background two regions are calculated through the traversal method, the threshold value with the maximum variance is selected as the threshold value of the whole image, the method is simple to calculate, and is not easily affected by external light, for the filtered image M*N to be segmented, the threshold value is set as T, the average gray value of the whole image is μ, the inter-class variance value is g, the number of foreground pixel points is N0, the proportion of the whole image pixel points is W0, the average gray value is μ0; the number of background pixel points is N1, the proportion of the whole image pixel points is W1, and the average gray value is μ1, then:

[0090]

[0091]

[0092] N0+N1=M*N

[0093] w0+w1=1

[0094] μ=w0*μ0+w1*μ1

[0095] g=w0*(μ0-μ) 2 +w1*(μ1-μ) 2

[0096] Further, the traversal data set g is obtained:

[0097] g=w0*w1(μ0-μ1) 2

[0098] The threshold value T when g is maximum is the image threshold value of the filtered image.

[0099] S2, the edge of the pretreated image is extracted based on the direction of the multi-resolution gray morphology, and the contour image of the glass to be measured is obtained; specifically, the following steps are included:

[0100] S201, constructing an edge model of the preprocessed image:

[0101] f(x, y) = b + a(xcosθ + ysinθ);

[0102] wherein b is the background gray scale of the preprocessed image, a is the profile amplitude of the preprocessed image, and θ is the direction of the profile.

[0103] The profile of the preprocessed image can be synthesized by combining the profiles extracted in different directions. In this embodiment, g(f(x, y), θ) represents the profile extracted in the direction θ, and the final profile is synthesized as follows:

[0104]

[0105] Considering the symmetry of the interval [0, л] and the interval [л, 2л], the above formula is rewritten as:

[0106]

[0107] S202, using a one-dimensional morphological operator to extract edges in the 0 direction and the π / 2 direction of the preprocessed image to obtain the longitudinal profile and the transverse profile; using a one-bit morphological operator with directionality to extract edges in multiple angles, and in order to avoid complex interpolation operations on the image, the directions can be 0, π / 4, π / 2, and 3π / 4. In this embodiment, the preprocessed image of the glass only needs to be registered by coordinate translation, so only the transverse profile and the longitudinal profile of the preprocessed image are extracted, i.e., edges in the 0 direction and the π / 2 direction of the preprocessed image are extracted; and when extracting the transverse profile, the influence of the longitudinal profile should be minimized, and vice versa; therefore, a relatively small structure operator is selected, and a 1x3 operator and a 3x1 operator are used in the 0 direction and the π / 2 direction to extract the longitudinal profile and the transverse profile, respectively.

[0108] S203, synthesizing the longitudinal profile and the transverse profile to obtain the profile image of the glass to be measured. Then, the image profiles in different directions are synthesized to obtain the profile image to be measured of the glass to be measured.

[0109] In this embodiment, the image secondary search registration algorithm based on multi-resolution cascade spatial alignment is used to match the profile image and the template image, including primary registration and secondary registration, wherein the purpose of the primary registration is to obtain the correspondence between the profile of the glass to be measured and the template profile points, and the overall similarity of the object is compared. The glass size is large, and it takes a long time to calculate the similarity of the entire image, and industrial production requires strong real-time performance, so a faster search method is needed. Specifically:

[0110] S3, based on the multi-resolution hierarchical progressive image search strategy, the to-be-tested contour image and the template contour image are registered once, and the recognition result is output and the pose information of the to-be-tested glass is obtained; the number of layers of the resolution is mainly determined by the object to be recognized, and the size of the resolution changes in a gradient form, and it is necessary to ensure that the relevant structure of the target object image in the highest layer of the resolution can be recognized. The recognition result is the image result of the to-be-tested glass, that is, the overlapping area of the contour of the to-be-tested glass and the template contour image is obtained, and the information of the center of the glass, that is, (x, y, θ) is obtained.

[0111] S301, the template contour image and the to-be-tested contour image are layered according to the size of the resolution, and the similarity measure between the corresponding layers of the template contour image and the to-be-tested contour image is calculated in sequence: the size of the resolution is determined according to the object to be recognized, and it is necessary to ensure that the relevant structure of the target object image in the highest layer of the resolution can be recognized, and the upper limit is determined by the resolution of the camera. Specifically, the to-be-tested contour image and the template contour image are converted into a point set a(x, y) in a constant division manner, at this time, the point set is a floating point type data, which is converted into an integer A(x, y), and the length is n. The similarity measure between the corresponding layers of the template contour image and the to-be-tested contour image is calculated:

[0112]

[0113] Wherein, P(x, y) = G(f(x, y)) is the edge point set of the to-be-tested glass contour image, A(x, y) is the edge point set of the template glass image contour, and n is the number of to-be-tested glass contour points;

[0114] When the calculated similarity measure reaches the minimum value, it means that the registration is successful, that is, the to-be-tested contour image and the template contour image are successfully registered.

[0115] S302, the calculated value of the similarity measure is compared with the preset value, if the calculated value is less than the preset value, the matching fails; if the calculated value is greater than or equal to the preset value, the matching succeeds, and the recognition result (x, y, θ) is output; in this embodiment, the first initial overall complete matching is performed at the highest layer of the ladder, if the matching succeeds, the matching information obtained is mapped to the next layer for search matching, if the matching fails, the search is stopped, until all layers are successfully searched and matched, and the corresponding matching information is output. One of the main parameters of image gradient matching is the similarity measure between the template image and the glass image to be recognized, if the matching similarity measure value is greater than or equal to the set similarity measure value, it is considered that the matching succeeds; if the matching similarity measure value is less than the set similarity measure value, it is considered that the matching fails. When the image gradient increases by one layer, the search speed can also be improved by 16 times, and under the condition of meeting the requirements, appropriately increasing the number of ladder layers is beneficial to enhancing the real-time performance of the target object recognition.

[0116] In this embodiment, when extracting the glass profile image to be measured, a Gaussian filter is used to obtain a smooth curve. In order to avoid image translation in the image step, the order size of the Gaussian filter must be even, such as 2*2, 4*4, 6*6, etc. In addition, the glass recognition method must be real-time, and has a high requirement on speed. Finally, a 4*4 Gaussian filter is used in the process of constructing the step,

[0117] Figure 5 the image step (1-4 layers) of the template profile image, Figure 6 the image step (2-5 layers) of the profile image to be measured. The fifth layer image in the image step of the profile image to be measured is very blurred compared with the template profile image. Therefore, starting from the fourth layer of the profile image to be measured, the matching is searched, the matching point coordinates found by direct search are multiplied by 2 to enter the next layer to continue searching for matching, and the matching is searched layer by layer until the matching of the bottom layer is successful. The matching result (x, y, θ) is obtained.

[0118] S303, the centroid coordinates of the glass to be measured and the rotation angle of the glass to be measured are obtained according to the recognition result (x, y, θ), and are output. Specifically, the position-related information is converted and sent to the I / O control end, that is, the related pose information of the glass to be measured is obtained, that is, positioning, which is convenient for subsequent grabbing and placing actions of the mechanical arm. The shape center x coordinate, y coordinate and rotation angle θ of the glass to be measured are the main position information on the two-dimensional image.

[0119] Because the centroid of the profile image to be measured can reflect the overall distribution information of the target object to a certain extent, the position information is obtained with the centroid of the graph as the center coordinate. Assuming that "1" represents the pixel value of the glass region to be measured in the image, and "0" represents the image background, the target object region can be represented by G, (x c , y c ) can be used to represent the coordinate position of the centroid, and it can be represented by the first moment:

[0120]

[0121] wherein, M 10 , M 01 represents the first moment, that is, the centroid of the image region, and M 00 represents the zero moment, that is, the total area of the image region.

[0122] M pq can be used to represent the (p+q)th moment of the image region g(x, y):

[0123]

[0124] Wherein, g(i, j) represents the pixel value of the target object region in the image, p represents the row number, q represents the column number, and G represents the target object region.

[0125] The angle of the to-be-tested glass is replaced by the difference between the rotation angle of the to-be-tested contour image and the template contour image. In the image registration part, θ of the to-be-tested contour image has been obtained, so the rotation angle of the to-be-tested glass is represented by θ.

[0126] The offset of the centroid point coordinates and the rotation angle of the to-be-tested glass are fed back to the mechanical arm, and then the mechanical arm is positioned and grabbed through the feedback value, and the grabbed parts are stacked.

[0127] S4, based on the contour camber vector similarity, the to-be-tested contour image and the template contour image are registered again, and the defects in the to-be-tested contour image are positioned and identified.

[0128] S401, the contours in the to-be-tested contour image and the template contour image are equally divided to obtain multiple contour segments; it is assumed that the contour of the to-be-tested contour image is represented by a point sequence:

[0129] p = {p i , i = 1, 2,..., N} i , i = 1, 2,..., N} i

[0130] Wherein, i represents the serial number of the contour point in the to-be-tested contour image, and N represents the number of contour points in the to-be-tested contour image.

[0131] The contour of the to-be-tested contour image is divided into k contour segments by N, and the length of the kth contour segment of the to-be-tested contour image is defined by the following formula:

[0132] SegLen(k) = ∑||p i -p i+1 ||2, i = s(k), s(k)+1,..., s(k+1)-1

[0133] Wherein, P i represents a point on the contour, and s(k) represents the starting contour point serial number of the kth contour segment of the target object contour.

[0134] S402, the camber function amplitude of all points in each contour segment is calculated to obtain the camber vector of each contour segment; P i (x i ,y i ) represents a point on the contour, it is assumed that s represents the starting contour point serial number of the kth contour segment of the target object contour, and t can be used to represent the terminal contour point serial number, so the starting contour point can be represented by P s (x s ,y s ​) can be represented by P t (x t ,y t ) and P j (x j ,y j ) respectively, P s (x s ,y s ) and P j (x j ,y j ) represent any point j (j = s + 1, s + 2,, t - 2, t - 1) in the contour segment of the target object contour except the start contour point and the end contour point, P j (x j ,y s ) and P s (x s ,y j ) are the two points on the straight line, and the distance from all points in the contour segment of the target object contour to the straight line is taken as the geometric feature of the precise matching of the arch height function AH, and the amplitude of the arch height function AH is calculated by the following formula:

[0135]

[0136] wherein,

[0138] AH j The amplitude of AH j can be represented by |AH s |, which does not contain positive and negative information, and its physical meaning is the distance from the contour point j to the straight line on which P s (x s ,y j ) and P j (x j ,y j ) are located. In order to more specifically describe the relative position information of the contour points of the to-be-tested contour image, the positive and negative information of AH s is also needed, and the positive and negative sign S of AH j is defined by the following formula:

[0139]

[0140] wherein, V t represents that AH j takes a positive value, V j represents that AH s takes a negative value, S represents the positive and negative of the non-zero element in the vector, x j represents the horizontal coordinate of any point in the contour segment of the to-be-tested object contour except the start contour point and the end contour point, x s represents the horizontal coordinate of the start contour point in the contour segment of the to-be-tested object contour, y tx-coordinate of the end profile point in the profile segment of the profile of the object to be measured t y-coordinate of the end profile point in the profile segment of the profile of the object to be measured

[0141] In the above formula, sign represents the sign of the non-zero element in the vector, V t and the product of V s The x-coordinate and the y-coordinate are both zero. In order to make the arch height function scale-invariant, the following formula is used:

[0142] AH' j = AH j / ‖p t -p s ‖2

[0143] where p t represents the end profile point in the profile segment of the profile of the object to be measured, and p s represents the start profile point in the profile segment of the profile of the object to be measured.

[0144] S403, the similarity value between the arch height vectors of the corresponding profile segments of the profile image to be measured and the template profile image is calculated.

[0145] The primary registration measures the similarity of the target object profile as a whole, and the mean square distance between the template profile image A(x, y) and the profile image to be measured P(x, y) is used as the similarity measure. The secondary registration uses the similarity of the arch height vectors to represent the similarity between the profile segments. Since there is a difference between the similarity values of the two, the average of the two is taken. Finally, the overall similarity ST of the target profile is described by the following formula:

[0146]

[0147] s represents the primary registration similarity measure, SG(i) represents the secondary registration similarity value, the template profile arch height vector H is [f1,…f N ], the profile arch height vector G of the glass to be measured is [g1,…g N ], AL(j) is the normalized length, and C is the profile circumference.

[0148] S405, the calculated value of the similarity value is compared with the preset value. If the calculated value is greater than the preset value, a rectangular mark is used to mark the defect.

[0149] Defect positioning is to determine the position of the centroid of the defect on the edge of the glass. The position in the original image is marked by a rectangle. In order to reduce the complexity of the algorithm, the centroid can be determined by the intersection point of the diagonals of the smallest rectangle enclosing the defect. Due to the randomness of the direction of the position of each defect in the image, the marking rectangle is a rotatable smallest enclosing rectangle.

[0150] Let P(x, y) be any point in the single defect set, in the coordinate system X-Y, if the rough edge frame rectangle is the minimum enclosing rectangle that can rotate, as shown in Figure 9 The angle β is defined as the angle between the first side of the minimum enclosing rectangle and the horizontal line when the horizontal X axis is rotated counterclockwise. For the new coordinate system XOY established by the minimum enclosing rectangle, any angle β in the new coordinate system X (OB) and Y (PB) has the following formula:

[0152]

[0154] The coordinate transformation is:

[0155]

[0156] The minimum rectangular area with angle β is:

[0157] A = (max(X) - min(X)) * (max(Y) - min(Y))

[0158] Find the minimum area corresponding to the coordinates that satisfy the above formula, and the four vertices of the rectangle in the new coordinate system are [min(X), min(Y)], [max(X), min(Y)], [max(X), max(Y)], and [min(X), max(Y)]. Then the inverse coordinate transformation can obtain the rectangle vertices in the original xOy coordinate system.

[0159] In the original coordinate system, x is OE and y is PE, which has the following formula:

[0160]

[0161] The inverse coordinate transformation is:

[0162]

[0163] After obtaining the rectangle vertices, calculate the centroid coordinates of each contour, and then the precise positioning can be realized. The minimum enclosing rectangle with the rotation angle β is the marking curve. As shown in Figure 10-20 The marking positioning results of several glass edge defects are shown. Further, by judging the geometric size of the marking rectangle, it can be determined whether it meets the rejection standard.

[0164] Example 2

[0165] A glass edge defect detection and position recognition system includes a control device, a conveyor device for conveying the glass to be tested, and an acquisition device for capturing images of the glass to be tested. The conveyor device includes a support frame and multiple conveyor rollers 5 rotatably mounted on the support frame. The multiple conveyor rollers 5 are arranged along the conveying direction of the glass to be tested, and each conveyor roller 5 is covered with a rubber ring distributed along its axial direction. The outer walls of the rubber rings are flush with each other to form a bearing surface for supporting the glass. In this embodiment, a photoelectric encoder is installed on one of the conveyor rollers 5, and the photoelectric encoder rotates synchronously with the conveyor roller 5 to collect the conveying speed of the glass and transmit it to the control device.

[0166] The acquisition device includes a gantry 2 and a camera 1 arranged above the conveyor. The number of cameras 1 can be 1-3. In this embodiment, the number of cameras 1 is 2. The resolution of the camera 1 is greater than or equal to 8192, the line frequency is greater than or equal to 10K, and a collection area is formed below the camera 1; each camera 1 has a lens with a focal length of 60mm, a thread M39╳1 / 26", and an aperture of 4.0. Specifically, the camera 1 is fixed to the gantry 2 by a first bracket, and the first bracket can achieve fine adjustment of three degrees of freedom, namely pitch, tilt and rotation. The gantry 2 is fixed with a bracket for mounting. The second bracket 3 holds the light source 4. The distance between the light source 4 and the conveyor roller 5 below it is 200-300mm. The light source 4 is long and slightly longer than the width of the roller, approximately 3.5-7m. Different lengths are selected according to the roller width. The light source 4 is a coaxial parallel light source 4, using a high-power, high-brightness LED, emitting full-spectrum white light with a color temperature of 3000-6500K. The light is distributed through a special focusing lens to achieve a parallel beam effect. The light reaches the surface of the object at the same angle, eliminating shadows caused by surface unevenness, improving image clarity, and evenly illuminating the surface.

[0167] The first bracket is further defined as follows: Figures 26-28 As shown, the first bracket includes a housing 6 fixedly connected to the gantry. A mounting plate 7 is fixedly connected to the housing 6. The mounting plate 7 is parallel to the side wall of the housing 6 and is connected by bolts. The camera 1 is mounted on the mounting plate 7 via a connecting plate 8. Specifically, the connecting plate 8 is located between the camera 1 and the mounting plate 7, and the bottom of the connecting plate 8 is rotatably connected to the mounting plate 7. The top of the connecting plate 8 is provided with a spring 10 for pulling the connecting plate 8 toward the mounting plate 7. One end of the spring 10 is fixedly connected to the mounting plate 7, and the other end of the spring 10 is fixedly connected to the connecting plate 8. A pad is fixedly connected to the top of the mounting plate 7. The pad is located between the mounting plate 7 and the connecting plate 8. The connecting plate 8 is provided with a knob 11. One end of the knob 11 passes through the connecting plate 8 and contacts the pad. When the knob 11 is rotated, the angle between the mounting plate 7 and the connecting plate 8 can be adjusted, thereby adjusting the position of the camera 1. The bottom wall of the housing 6 is provided with a shooting port 9 for the camera 1 to shoot.

[0168] The conveying device further comprises a third support 12 fixed on the ground, and the conveying roller 5 is rotatably arranged on the third support.

[0169] The gantry 2 is provided with a plurality of photoelectric switches, and the number of the photoelectric switches can be 1, 2 or 4. The number of the photoelectric switches is related to the width of the roller way (the plurality of conveying rollers 5 are distributed along the conveying direction to form the roller way). When the number of the photoelectric switches is 1, the photoelectric switch is located at the center of the roller way. When the number of the photoelectric switches is 2, the two photoelectric switches are respectively located at 1 / 4 and 3 / 4 of the roller way. When the number of the photoelectric switches is 4, the four photoelectric switches are respectively located at 1 / 6, 1 / 3, 2 / 3 and 5 / 6 of the roller way. In the embodiment, the number of the photoelectric switches is 2. When the leading edge of the glass to be measured enters the collection area, the photoelectric switch at the position controls the camera 1 to start collecting the image of the glass to be measured. When the trailing edge of the glass to be measured leaves the collection area, the photoelectric switch at the position controls the camera 1 to end collecting the image of the glass to be measured.

[0170] The distance between the adjacent two conveying rollers 5 in the collection area is 150-200 mm, and the distance between the adjacent two rubber rings on the conveying roller 5 in the collection area is 100-150 mm, so as to stabilize the glass and prevent vibration. In addition, the color of the roller is dark gray and matt, so as to prevent stray light interference. The distance between the adjacent two conveying rollers 5 on the two sides of the collection area is 200-250 mm, and the distance between the adjacent two rubber rings on the conveying roller 5 in the area is 150-200 mm.

[0171] The control device comprises an image collection module, a signal collection module, a defect detection module and a pose recognition module. The signal collection module is used to collect the conveying speed of the glass to be measured and control the switching of the camera 1. The image collection module is used to receive. Specifically, the control device further comprises a drive-by-wire communication module, a light source 4 control module, a statistical analysis module, a man-machine interaction module and an overall logic flow.

[0172] The drive-by-wire communication module is used to control the conveying device to realize the conveying of the glass to be measured. The drive-by-wire communication module sends data packet information to the system. The system detects the photoelectric switch, judges the arrival of the glass to be measured, controls the light source 4 to irradiate the surface of the glass to be measured, measures the conveying speed through the encoder and the collection card and triggers the camera 1 to collect the image. The image collection module obtains the image data and processes it. The defect detection module and the pose recognition module judge the defect and recognize the pose. Then, the results are fed back to the drive-by-wire system (the drive-by-wire system is the control center of the entire glass production line) through the drive-by-wire communication module. The drive-by-wire system completes the subsequent processing of rejection and mechanical hand grabbing and stacking. In order to set parameters, issue commands, display working states, perform statistical analysis and the like, a man-machine interaction interface is designed. The system hardware and software are organically combined through the overall logic flow to complete the functions of the entire system.

[0173] The line control communication module realizes data exchange between the control device and the PLC line control system through the network UDP protocol and the line control communication protocol. The measurement and control program sends the detected defect type, size, position, and glass pose information to the PLC. After the PLC analyzes, the following processing is performed:

[0174] (1) If there is a defect.

[0175] Notify the subsequent process to abandon the processing of the glass sheet and perform scrap.

[0176] (2) If there is no defect or the defect can be ignored.

[0177] Notify the subsequent process (paper laying / powder spraying or stacking) to position according to the glass plate size, pose, etc. Control the suction cup at the front end of the manipulator to make the suction cup parallel to the long side of the glass plate and at the center position of the glass plate rectangular area. The manipulator performs correct stacking operation according to the glass plate grabbing pose.

[0178] The light source control module controls the opening of the light source 4 before collecting the original image and controls the closing of the light source 4 after image processing. According to the current working condition (glass type, mechanical process, etc.), the brightness of the light source 4 is automatically adjusted to achieve the best imaging effect.

[0179] The signal acquisition module acquires encoder pulses and photoelectric switch signals through the encoder data acquisition card interface to realize speed measurement, external triggering of the camera 1, and photoelectric switch signal judgment.

[0180] The image acquisition module realizes the configuration, trigger control, image acquisition, and image data transmission of the camera 1 through the development of the camera 1 data packet (SDK).

[0181] The defect detection module and the pose recognition module process the glass plate image data to determine the defect type, size, position, and glass pose information. To accurately obtain the defect position and glass pose, the camera 1 needs to be calibrated to obtain the corresponding relationship between the image pixel coordinates and the physical coordinates.

[0182] The statistical analysis module saves the defect information, pose information, state, and alarm information to the database. The program will perform statistics on various types of information to obtain short-term and daily, weekly, monthly, and annual statistical results, and display them in the form of reports or historical curves, etc. for cold end equipment maintenance, analysis comparison, result evaluation, etc.

[0183] The human-computer interaction module provides an interface for users to interact with the measurement and control program. Users can set parameters, issue commands, calibrate the camera 1, view working status, obtain alarm information, view glass images, curves, and roller running speed, view image processing results, and perform statistical analysis, etc.

[0184] The overall logic flow organically links the various modules, sequentially, time-divisionally and multi-taskingly processes the working procedure of the system, and conforms to the logic procedure of the equipment operation.

[0185] The control device further comprises a measurement and control program, which is a graphical user interface program developed under the Visual Studio environment based on the Windows operating system, and mainly includes: hardware setting, defect judgment rule setting, camera 1 debugging, camera 1 correction, image display, curve display, roller speed display, processing result display, offline data file analysis, alarm viewing, state display and other parameter setting, etc. In addition to the image display, curve display, roller speed display, processing result display and offline data file analysis shown on the main interface, it further includes the parts as shown in Figures 40-48 .

[0186] In this embodiment, the control device comprises a signal interface mechanism, the signal mechanism provides a 3-way encoder pulse high-speed input interface, a 4-way photoelectric switch signal low-speed input interface, a 1-3 camera 1 power supply interface and a DC / DC conversion interface. It has overvoltage, overcurrent protection and anti-high-frequency interference functions. Specifically:

[0187] 1) The encoder pulse high-speed input interface, one pulse (A+) of the encoder is applied to the photoelectric isolation input stage after comparison, and the photoelectric isolation output stage is shaped and output in two ways:

[0188] First way: to the external trigger control end of camera 1 to control the exposure of camera 1.

[0189] Second way: A+ and another pulse B+ to the encoder data acquisition card to detect the conveying speed and control the logic flow.

[0190] The comparator (U1) of the input stage can effectively eliminate high-frequency interference pulses by adjusting the reference voltage (P1).

[0191] The photoelectric isolation (U2) can effectively isolate the input and output electrical signals, and has electrical insulation and common mode rejection capability.

[0192] 2) The photoelectric switch signal low-speed input interface, one way (P1) of the photoelectric switch signal is output to the GPIO interface of the encoder data acquisition card after photoelectric isolation and comparison shaping.

[0193] 3) The camera 1 power supply interface provides 24VDC power supply for 1-3 cameras 1. The external trigger signal of camera 1 is also connected through this interface.

[0194] 4) DC / DC conversion interface, 24VDC from switching power supply, after filtering, access to 24V / 12V isolation conversion circuit (U5, U51), get the camera 1 trigger and opto-isolated switch isolation required 12VDC voltage, 12VDC through the isolation conversion circuit (U6), get the opto-isolated module required 5VDC voltage.

[0195] Power supply circuit string from the self-resuming fuse (F1), play the role of over-current protection. Power supply both ends across TVS diode (D2) play the role of over-voltage inclusion.

[0196] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for glass edge defect detection and pose recognition, characterized in that, The method comprises the following steps: S1, collecting an original image of the glass to be measured and performing preprocessing to obtain a preprocessed image; S2, performing edge extraction on the preprocessed image based on a direction-based multi-resolution gray-scale morphology to obtain a profile image of the glass to be measured; S3, performing one-time registration on the profile image of the glass to be measured and a template profile image based on a multi-resolution hierarchical progressive image search strategy, outputting a recognition result and obtaining pose information of the glass to be measured; S4, performing secondary registration on the profile image of the glass to be measured and the template profile image based on profile camber vector similarity, and positioning and identifying defects in the profile image of the glass to be measured.

2. The method of claim 1, wherein: In the S1, the preprocessing of the original image comprises the following steps: S101, performing filtering processing on the original image by using a Gaussian bilateral filter to obtain a filtered image; S102, performing threshold segmentation on the filtered image by using an Otsu method to obtain the preprocessed image.

3. The method of claim 1, wherein: In the S2, the profile image acquisition method comprises the following steps: S201, constructing an edge model of the preprocessed image; S202, performing edge extraction on the preprocessed image in 0 direction and π / 2 direction by using a one-dimensional morphological operator to obtain a longitudinal profile and a transverse profile; S203, synthesizing the longitudinal profile and the transverse profile to obtain the profile image of the glass to be measured.

4. The method of claim 3, wherein: In the S201, the edge model is: f(x,y)=b+a(xcosθ+ysinθ); Wherein, b is the background gray value of the preprocessed image, a is the profile amplitude of the preprocessed image, and θ is the direction of the profile.

5. The method of claim 1, wherein: The S3 comprises the following steps: S301, layering the template profile image and the profile image of the glass to be measured according to the resolution size of the profile image of the glass to be measured, and sequentially calculating the similarity measure between the corresponding layers of the template profile image and the profile image of the glass to be measured: Wherein, P(x,y)=G(f(x,y)) is the edge point set of the profile image of the glass to be measured, A(x,y) is the edge point set of the profile image of the template glass, and n is the number of profile points of the glass to be measured; S302, comparing the calculated value of the similarity measure with a preset value, if the calculated value is less than the preset value, the matching fails, if the calculated value is greater than or equal to the preset value, the matching succeeds, and a recognition result (x,y,θ) is outputted; S303, obtaining the centroid point coordinates of the glass to be measured and the rotation angle of the glass to be measured according to the recognition result (x,y,θ), and outputting.

6. The method of claim 1, wherein: The S4 comprises the following steps: S401, equally dividing the profiles in the profile image of the glass to be measured and the template profile image to obtain multiple profile segments; S402, calculating the camber function amplitude of all points in each profile segment to obtain the camber vector of each profile segment; S403, calculating the similarity value between the camber vectors of the corresponding profile segments of the profile image of the glass to be measured and the template profile image; S405, comparing the calculated value of the similarity value with a preset value, if the calculated value is greater than the preset value, a rectangular mark is used to mark the defect.

7. The method of claim 6, wherein: In the S402, the camber function amplitude is: wherein, x t is the horizontal coordinate of an arbitrary point in the profile segment of the profile of the object to be measured, x s is the horizontal coordinate of the starting profile point in the profile segment of the profile of the object to be measured, x j is the horizontal coordinate of an arbitrary point in the profile segment of the profile of the object to be measured, x s is the horizontal coordinate of the starting profile point in the profile segment of the profile of the object to be measured, y t is the vertical coordinate of the ending profile point in the profile segment of the profile of the object to be measured, y s is the vertical coordinate of the starting profile point in the profile segment of the profile of the object to be measured, y j is the vertical coordinate of an arbitrary point in the profile segment of the profile of the object to be measured, y P t is the ending profile point in the profile segment of the profile of the object to be measured, P s is the starting profile point in the profile segment of the profile of the object to be measured.

8. A glass edge defect detection and pose identification system, comprising: The control device, the conveying device for conveying the glass to be tested and the acquisition device for acquiring the image of the glass to be tested, the acquisition device comprises a camera (1) arranged above the conveying device, so that the camera (1) forms an acquisition area below; the control device comprises an image acquisition module, a signal acquisition module, a defect detection module and a pose recognition module, the signal acquisition module is used for acquiring the conveying speed of the glass to be tested and controlling the switching of the camera (1), and the image acquisition module is used for receiving the original image.

9. A glass edge defect detection and pose identification system as claimed in claim 8, characterized in that: The conveying device comprises a support frame and a plurality of conveying rollers (5) rotatably arranged on the support frame, the plurality of conveying rollers (5) are distributed along the conveying direction of the glass to be tested, and each conveying roller (5) is sleeved with a rubber ring distributed along the axial direction thereof.

10. The glass edge defect detection and pose identification system of claim 9, wherein: The distance between the adjacent two conveying rollers (5) in the acquisition area is 150-200 mm, and the distance between the adjacent two rubber rings on the conveying roller (5) in the acquisition area is 100-150 mm.

Images