Defect detection method based on Zernike orthogonal moment and Hash algorithm

Abstract

The invention belongs to the field of defect detection, and particularly relates to a defect detection method based on Zernike orthogonal moments and a Hash algorithm. The invention aims to solve the problem of low detection efficiency of the existing electronic component dispensing defect detection technology. According to the method, a mode of realizing image enhancement through a fuzzy set is established, and the integrity of information can be guaranteed while the region of interest is highlighted. And aiming at different colloid defects, different detection methods and detection areas are considered to carry out judgment. Through a Zernike orthogonal moment method, detection areas on the inner side and the outer side of a dispensing groove are obtained, and calculation time can be effectively shortened by replacing an original image with Hash of the image for calculation. And meanwhile, an algorithm for reducing the area of a to-be-matched region through approximation is established on the basis of an average hash algorithm, so that the time required by matching calculation can be effectively shortened, the operation efficiency can be greatly improved by combining a zero-mean normalized product correlation algorithm, and the efficiency of detecting the dispensing defect of the electronic element is improved.

Description

Technical Field

[0001] The invention belongs to the field of defect detection, and in particular relates to a defect detection method based on Zernike orthogonal moments and a hash algorithm. Background Art

[0002] With the rapid development of the semiconductor and electronics industries, the manufacturing industry has continuously increased its requirements for production efficiency and precision. The traditional production model that relies on manual operation can no longer meet current needs. Manual operation is prone to errors, which affects production accuracy. The emergence of machine vision technology solves this problem. It can detect and classify different types of products in real time, reducing the impact of fatigue or environmental factors on manual labor.

[0003] Machine vision technology is not only widely used in industrial production for detail and defect detection, but also plays an important role in agriculture, medicine, and the military. In the manufacture of electronic components, the dispensing process is crucial, involving the precise application of materials such as UV glue and thermal conductive glue. Too much or too little colloid in the dispensing process will affect product performance and cause quality errors. Traditional manual inspection has limited scope and misjudgment problems, while machine vision can efficiently and accurately detect dispensing defects and improve production quality and efficiency.

[0004] For defects in glue-dispensing electronic components, including but not limited to uneven glue, poor glue flow, incomplete glue curing, etc., due to the uneven glue dispensing, the glue-dispensing electronic components will have uneven thickness of the glue and the presence of small holes and gaps. Therefore, the electronic component glue dispensing defect detection method will cause the loss of image information during the image enhancement process. In order to solve the problem of image information loss during the image enhancement process, extra time will be spent, resulting in the problem of low detection efficiency in the existing electronic component glue dispensing defect detection technology. Summary of the invention

[0005] The invention aims to solve the problem of low detection efficiency of existing electronic component dispensing defect detection technology.

[0006] S1: Collecting images to be detected; the images to be detected include images of original parts without glue dots and images of original parts with glue dots;

[0007] S2: Generate a matching image and a matching template according to the image to be detected;

[0008] S3: Perform template matching based on the matching image and the matching template to obtain the sub-region to be detected of the image to be detected.

[0009] S4: Perform defect detection on the sub-region to be detected of the image to be detected, obtain the detection result, and visualize the detection result using visualization software.

[0010] The beneficial effects of the present invention are:

[0011] The present invention establishes a fuzzy set to achieve image enhancement, which can highlight the area of ​​interest while ensuring the integrity of information. Different detection methods and detection areas are considered for different colloid defects. The detection areas on both sides of the dispensing groove are obtained by the Zernike orthogonal moment method, which improves the accuracy of subsequent insufficient glue detection.

[0012] A template matching method based on hash algorithm is proposed. According to the properties of image hash algorithm, the input image is mapped into a digital sequence, which is called the hash value of the image. Using the hash of the image instead of the original image for calculation can effectively reduce the calculation time. At the same time, based on the average hash algorithm, an algorithm that reduces the area of ​​the area to be matched by approximation can effectively reduce the time required for matching calculation. Combined with the zero-mean normalized product correlation algorithm, it can greatly improve the operation efficiency, while also ensuring the accuracy of matching, and improving the efficiency of electronic component dispensing defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the dispensing detection process of the present invention;

[0014] Figure 2 It is a schematic diagram of extracting the colloid area of ​​the dispensing electronic component of the present invention;

[0015] Figure 3 It is a schematic diagram of the effects of several filtering methods of the present invention;

[0016] Figure 4 Schematic diagrams of the effects of several different image enhancement methods of the present invention;

[0017] Figure 5 It is a schematic diagram of the colloid region obtained after the threshold segmentation of the present invention;

[0018] Figure 6 It is a schematic diagram of the area to be detected after excluding the interference area of ​​the present invention;

[0019] Figure 7 This is a schematic diagram of the edge model under an ideal state of the present invention;

[0020] Figure 8 It is a grayscale schematic diagram of the present invention;

[0021] Fig. 9 is a schematic diagram of a binary matrix after processing of the present invention;

[0022] Fig.10 It is a schematic diagram of the detection results of different defect images using the average hash algorithm of the present invention;

[0023] Fig.11It is a schematic diagram showing the detection results of the present invention. DETAILED DESCRIPTION

[0024] Specific implementation method 1: Combination Figure 1 The invention is described, comprising:

[0025] S1: Collecting images to be detected; the images to be detected include images of original parts without glue dots and images of original parts with glue dots;

[0026] The non-glue original image is an image collected before the electronic component is subjected to the glue dispensing process.

[0027] The dispensing original image is an image collected after the electronic component undergoes a dispensing process.

[0028] The glue dispensing process of electronic components refers to the process of applying, sealing, filling, and dripping UV glue, thermal conductive glue, etc. glue on the product to play the role of reinforcement, sealing, insulation, etc.

[0029] S2: Generate a matching image and a matching template according to the image to be detected;

[0030] S3: Perform template matching based on the matching image and the matching template to obtain the sub-region to be detected of the image to be detected.

[0031] S4: Perform defect detection on the sub-region to be detected of the image to be detected, obtain the detection result, and visualize the detection result using visualization software.

[0032] Specific implementation method 2: The difference between this implementation method and specific implementation method 1 is that:

[0033] The camera used to collect the image to be detected in S1 is a CMOS camera; the lighting method is front illumination, and the light source is an LED ring light source;

[0034] CMOS cameras can process light signal conversion and image processing in parallel, thereby increasing speed. In addition, CMOS only requires one power supply and consumes less power, which makes it more energy efficient. The lens and model of the CMOS camera are determined according to the target size of the electronic component to be tested.

[0035] For example, the target size of the electronic component in actual use of the present invention is 16.0mm×12.0mm, and the required accuracy is not less than 0.02mm. The detection target should occupy more than three quarters of the field of view to facilitate the detection of the integrity of the target. Therefore, the target area can be set to about 75% of the overall image area. According to calculations, a Basler CMOS camera model BasleracA3800-10gm is used, and the lens selected is a Basler lens model FL-HC1212B-VG.

[0036] The front LED light source means to use a front-illuminating LED light source.

[0037] The front illumination includes a strip light source, a coaxial light source, a square light source and a ring light source.

[0038] The strip light source is suitable for objects with larger square structures, and the color can be matched as required;

[0039] Coaxial light source can eliminate shadows caused by uneven surface of objects, and is suitable for objects with extremely high reflectivity;

[0040] The square light source is composed of four strip light sources, which can be controlled and positioned separately, with high flexibility;

[0041] Ring light sources can provide combinations of different illumination angles and colors, which can better reflect the three-dimensional information of objects and have a better effect on the detection of surface areas of objects.

[0042] Because the object to be detected in the present invention is an electronic component with a small size, and the detection is aimed at irregular and uneven colloids, the selection of an annular light source is conducive to better imaging.

[0043] The main light sources commonly used in machine vision are fluorescent lamps, halogen lamps and LEDs. Compared with other traditional light sources, LED lamps have advantages such as fast response speed, low energy consumption, small size, strong controllability and long life.

[0044] The other steps and parameters are the same as those in the first specific implementation.

[0045] Specific implementation method three: This implementation method is different from the specific implementation method one in that:

[0046] In S2, a matching image is generated according to the image to be detected; the specific process is as follows:

[0047] S2.1: obtaining an image of the original part without glue spots after extracting the area to be inspected according to the image of the original part without glue spots;

[0048] Obtaining an image of the dispensing original part after extracting the area to be inspected according to the dispensing original part image;

[0049] S2.2: extracting the original image of the non-dot glue after the area to be detected is used as a matching template;

[0050] Extract the image of the dispensing original after the area to be inspected to obtain a matching image;

[0051] In the step S2.1, the image of the original part without dots of glue is obtained after the area to be inspected is extracted according to the image of the original part without dots of glue;

[0052] The specific process of obtaining the dispensing original image after extracting the area to be detected based on the dispensing original image is as follows:

[0053] S2.1.1: performing bilateral filtering on the image of the original part without dots of glue to obtain the original image of the original part without dots of glue after bilateral filtering;

[0054] Performing bilateral filtering on the dispensing original image to obtain the dispensing original image after bilateral filtering;

[0055] S2.1.2: performing image enhancement processing on the image of the original part without dots of glue after bilateral filtering to obtain an enhanced image of the original part without dots of glue;

[0056] Performing image enhancement processing on the dispensing original image after bilateral filtering to obtain an enhanced dispensing original image;

[0057] S2.1.3: performing image threshold segmentation processing on the enhanced non-dot glue original image to obtain the non-dot glue original image after image threshold segmentation processing;

[0058] Performing image threshold segmentation processing on the enhanced dispensing original image to obtain the dispensing original image after image threshold segmentation processing;

[0059] S2.1.4: extracting the sub-pixel edge contour of the non-dot glue original image after the image threshold segmentation processing; obtaining the sub-pixel edge contour of the non-dot glue original image;

[0060] Extracting sub-pixel edge contours of the dispensing original image after the image threshold segmentation process; obtaining the sub-pixel edge contours of the dispensing original image;

[0061] S2.1.5: Obtain the area to be detected according to the sub-pixel edge contour of the original image without glue dot and the sub-pixel edge contour of the original image with glue dot;

[0062] Extracting the enhanced non-dot glue original image according to the area to be detected, and obtaining the non-dot glue original image after extracting the area to be detected;

[0063] The enhanced dispensing original image is extracted according to the area to be detected, so as to obtain the dispensing original image after the area to be detected is extracted; the other steps and parameters are the same as those in one of the specific implementation modes one to two.

[0064] Specific implementation method 4: This implementation method is different from specific implementation methods 1 to 4 in that:

[0065] The S2.1.1 uses bilateral filtering, which is a nonlinear image processing technique that can smooth an image while retaining edge information. Unlike traditional Gaussian filtering, bilateral filtering not only considers the spatial distance between pixels, but also the difference between pixel values, so it can effectively eliminate noise and protect image edges.

[0066] The expression formula of bilateral filtering is as shown in the formula:

[0067]

[0068] In the formula, ω(x,y,m,n) represents the weighting coefficient, which is expressed by the following formula:

[0069] ω(x,y,m,n)=d(x,y,m,n)*r(x,y,m,n)

[0070] The spatial domain weighting coefficient is expressed by the following formula:

[0071]

[0072] The filter weight coefficient of the value range is expressed by the following formula

[0073]

[0074] exp represents the exponential function, f() represents the filter function, m represents the horizontal pixel mean, and n represents the vertical pixel mean. Represents the variance, which is obtained through the collected images. It can be seen that due to the influence of the collection environment and the camera, the collected images will have some noise, but the amount of noise is not large, and the main existing Gaussian noise.

[0075] The collected images are processed with different filtering algorithms to obtain Figure 3 The result image in the figure is processed by applying different filtering algorithms to the collected image. The results are shown in the table. From the data, we can know that after bilateral filtering, the PSNR value is the largest while eliminating noise, indicating that the processing effect is the best.

[0076]

[0077] In the above 2.1.2, the image enhancement processing is performed on the non-dot glue original image after bilateral filtering to obtain the enhanced non-dot glue original image; the image enhancement processing is performed on the dot glue original image after bilateral filtering to obtain the enhanced dot glue original image; the specific process is:

[0078] S2.1.2.1: Calculate the membership degree of each pixel of the bilaterally filtered point-free glue original image;

[0079] Calculate the membership degree of each pixel of the bilaterally filtered dispensing original image;

[0080] S2.1.2.2: Perform data enhancement processing on each pixel of the dot-free glue original image according to the membership degree of each pixel of the dot-free glue original image to obtain an enhanced dot-free glue original image pixel, and all enhanced dot-free glue original image pixels constitute an enhanced dot-free glue original image;

[0081] Data enhancement processing is performed on each pixel of the dispensing original image according to the membership degree of each pixel of the dispensing original image to obtain enhanced dispensing original image pixels, and all enhanced dispensing original image pixels constitute an enhanced non-dispensing original image.

[0082] The other steps and parameters are the same as those in the first to third embodiments.

[0083] Specific implementation method 5: This implementation method is different from specific implementation methods 1 to 4 in that:

[0084] The specific process of performing image threshold segmentation processing on one of the enhanced non-dot glue original images in S2.1.3 to obtain a non-dot glue original image after image threshold segmentation processing is as follows:

[0085] S2.1.3.1: Set the global threshold T,

[0086] The pixel value of the enhanced non-dot glue original image is set to 255 at the position where the pixel value is higher than the global threshold T;

[0087] The pixel values ​​of the positions in the enhanced non-dot glue original image where the pixel values ​​are not higher than the global threshold T are set to 0;

[0088] Finally, the original image of the non-point glue is obtained after preliminary segmentation processing;

[0089] The preferred setting process of the global threshold T is:

[0090] The calculation process is to calculate the gray distribution histogram of the enhanced non-dot glue original image, determine the pixels at the trough position according to the gray distribution histogram, and use the pixels at one of the trough positions as the global threshold T;

[0091] S2.1.3.2: Set the area threshold of the connected area.

[0092] Calculate the area of ​​the connected region in the pointless glue original image after the preliminary segmentation process.

[0093] The connected area refers to the area with the same pixel value;

[0094] For connected areas whose area is smaller than the connected area threshold, the pixel value is set to 255;

[0095] Finally, the original image of the glue without dots is obtained after the image threshold segmentation processing;

[0096] The image threshold segmentation process for the original image of the glue dot is the same as that for the original image without the glue dot;

[0097] In the step S2.1.4, sub-pixel edge contour extraction is performed on the dot-free glue original image after the image threshold segmentation process; and the dot-free glue original image after the sub-pixel edge contour extraction is obtained;

[0098] Sub-pixel edge contour extraction is performed on the dispensing original image after the image threshold segmentation processing; the dispensing original image after sub-pixel edge contour extraction is obtained, and the specific process is as follows:

[0099] The sub-pixel edge contour of the non-dot glue original image after image threshold segmentation is extracted by using the Zernike orthogonal moment method to obtain the sub-pixel edge contour of the non-dot glue original image.

[0100] The sub-pixel edge contour of the original image without glue dots after image threshold segmentation is extracted by using the Zernike orthogonal moment method, and the sub-pixel edge contour of the original image with glue dots is obtained.

[0101] Zernike orthogonal moments are a mathematical tool for describing image features known to those skilled in the art, and are suitable for analyzing circular or axisymmetric objects.

[0102] In sub-pixel edge detection, Zernike orthogonal moments can improve the accuracy and efficiency of edge detection through its rotation invariance. It is necessary to calculate the four edge parameters for detection.

[0103] The four parameters are the background gray value h, the step height k, the vertical distance l from the center to the edge, and the angle θ between the vertical line and the x-axis. Figure 7 This is an edge model diagram under ideal conditions. In the figure, L divides the ideal model into two areas with different grayscale values.

[0104] In the discrete state, the moment value is the convolution of the template and the image grayscale.

[0105] The Zernike orthogonal moment method is used to extract the sub-pixel edge contour of the non-dot glue original image after the image threshold segmentation process. The specific process of obtaining the sub-pixel edge contour of the dot glue original image is as follows:

[0106] S2.1.4.1: Assume that the sub-pixel edge contour of the dot-free collagen original image after image threshold segmentation is composed of M unit disks;

[0107] S2.1.4.2: Set the size of the template for sampling within the unit disk to N × N,

[0108] S2.1.4.3: Based on the template coefficients of the corresponding areas of the square and the disk, calculate N Zernike templates;

[0109] S2.1.4.4: Calculate the moment value of the Zernike orthogonal moment based on the N Zernike templates and the image of the pointless glue original after the image threshold segmentation processing, and determine whether it is an edge point based on the moment value of the Zernike orthogonal moment and the gray value parameter of the image of the pointless glue original after the image threshold segmentation processing;

[0110] The moment value of the Zernike orthogonal moment is the convolution value of the template and the current pixel. At this point, we know that the radius of the disk is N / 2, and the vertical distance l from the origin to the ideal edge should be N / 2 of the original. Assuming that the original image is represented by f(x,y), where the pixel coordinates are (x,y), we can

[0111] The calculation formula of sub-pixel coordinates (x', y') of the sub-pixel edge detection algorithm of Zernike orthogonal moments is:

[0112]

[0113] The template size used in the present invention is 7×7. The template is used to divide the disk. The square grid area in the i-th row and the j-th column is recorded as S i,j , the disk is recorded as C, and the template coefficient of the area corresponding to the square and the disk is obtained. After calculating N Zernike templates, the processed image is convolved with the N templates respectively to obtain the moment values ​​of N Zernike orthogonal moments. According to the distance parameter and gray value parameter, it is judged whether it is an edge point. It can be expressed as

[0114]

[0115] The area to be inspected in S2.1.5 includes an overflow inspection area and a insufficient inspection area;

[0116] In the S2.1.5, the area to be detected is obtained according to the sub-pixel edge contour of the non-dot glue original image and the sub-pixel edge contour of the dot glue original image; the specific process is:

[0117] S2.1.5.1: Fill the sub-pixel edge contour of the original image without glue to obtain the original glue-free area;

[0118] Fill the sub-pixel edge contour of the original glue-dispensing image to obtain the original glue-containing area;

[0119] S2.1.5.2: Perform expansion morphology treatment on the original rubber-containing area to obtain an expansion morphology-treated rubber-containing area;

[0120] Performing corrosion morphology treatment on the original glue-containing area to obtain a corrosion morphology treated glue-containing area;

[0121] S2.1.5.3: Perform differential processing on the original glue-free area and the original glue-dispensing part image after expansion morphology processing to obtain the glue overflow detection area;

[0122] The glue-less detection area is obtained by performing differential processing on the original glue-free area and the glue-dispensing original image after corrosion morphology processing.

[0123] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0124] Specific implementation method 6: This implementation method is different from specific implementation methods 1 to 5 in that:

[0125] The specific process of calculating the membership degree of a certain pixel of the image in S2.1.2.1 is as follows:

[0126]

[0127]

[0128]

[0129] Where x represents the pixel value, x1 represents the first pixel threshold, x2 represents the second pixel threshold, x3 represents the third pixel threshold, μ d (x) represents the dark pixel membership, μ g (x) represents the gray pixel membership, μ b (x) represents the bright pixel membership;

[0130] The membership degree of a certain pixel obtained in S2.1.2.2 is used to perform data enhancement processing on a certain pixel; the specific process is:

[0131]

[0132] where s d Indicates the value of a completely black pixel, s g Represents the grayscale pixel value, s b represents the all-white pixel value, and s represents the pixel value at the current position after data enhancement processing;

[0133] The first pixel threshold, the second pixel threshold, and the third pixel threshold can be determined by filtering the image to be enhanced and obtaining its histogram, based on three peaks obtained from the histogram, or calculated and determined based on historical data, or determined through human experience.

[0134] The histogram is a commonly used tool in image processing, which is used to represent the distribution of pixel grayscale values ​​in an image. It shows the frequency of occurrence of pixels of different grayscale levels (such as 0 to 255). The histogram obtained after the image collected by the present invention is generally only three peaks, so the preferred values ​​of the three pixel thresholds are the peak values ​​of the three frequency peaks;

[0135] The specific process of performing data enhancement processing on a certain pixel based on the membership degree of a certain pixel obtained in S2.1.2.2 is as follows:

[0136]

[0137] where s d Indicates the value of a completely black pixel, s g Represents the grayscale pixel value, s b represents the all-white pixel value, and s0 represents the pixel value at the current position after data enhancement processing. The grayscale pixel value is artificially set and is between all black and all white;

[0138] The other steps and parameters are the same as those in the first to fifth embodiments.

[0139] Specific implementation method 7: This implementation method is different from specific implementation methods 1 to 6 in that:

[0140] In S3, template matching is performed according to the matching image and the matching template to obtain the sub-region to be detected of the image to be detected. The specific process is as follows:

[0141] S3.1: Determine a minimum matching image size, and divide the matching image into N matching images according to the minimum matching image size, where N is a positive integer;

[0142] S3.2: Calculate the average hash value of N matching images to obtain N matching average hash values;

[0143] Calculate the average hash value of the matching template; obtain the average hash value of the matching template;

[0144] S3.3: Perform template matching based on the N matching average hash values ​​and the matching template average hash value to obtain a template matching result image.

[0145] S3.4: Obtaining the sub-region to be detected of the image to be detected according to the template matching result image;

[0146] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0147] Specific implementation eight: This implementation differs from specific implementations one to seven in that:

[0148] The specific process of determining the minimum matching size in S3.1 is as follows:

[0149] S3.1.1: Set a contrast image T1 with a template;

[0150] S3.1.2: Get k regions S of different matching sizes based on the contrast image T1 k , where S k The formula is expressed as follows

[0151]

[0152] Here M and N are the length and width of the contrast image T1;

[0153] Calculate T1 and S k The average hash value of T1 and S is calculated based on the average hash value. k The Hamming distance of; where, according to the image S i The resulting Hamming distance is denoted as d i ;

[0154] Determine the minimum matching image size based on the Hamming distance, the minimum matching image size S n The corresponding d n Satisfy the formula

[0155] d1>d2>…>d i >d i+1 >…>d n <d n+1

[0156] At this time n The corresponding S n That is, the minimum area where there is an image to be matched. The Hamming distance is the number of different characters in the corresponding positions of two strings of equal length. By obtaining two strings of equal length using two images of the same size, we can compare the number of difference values ​​of the two sets of strings, thereby obtaining the Hamming distance between the two, and thus judging the degree of similarity.

[0157] The specific process of calculating the average hash value in S3.2 is:

[0158] The template image and the image to be matched are scaled to 8×8 size and then grayed out to obtain the following Figure 8 The eight images in the image are converted into corresponding data matrices. The images are converted into corresponding data matrices.

[0159] The data matrix is ​​composed of the size of each pixel of the scaled grayscale image. The mean of all elements in the matrix can be obtained through the obtained data matrix, and the obtained mean can be compared with all the data in the data matrix.

[0160] In the above matrix, the elements greater than the mean are set to 1, and the elements less than the mean are set to 0, so that a matrix consisting of 1 and 0 can be obtained, as shown in the following figure: Fig. 9 The matrix shown in the figure constructs each matrix into a vector to obtain a 64-bit hash value

[0161] In S3.3, template matching is performed according to the N matching average hash values ​​and the matching template average hash value to obtain a matching result image of the template matching. The specific process is as follows:

[0162] S3.3.1: Calculate the Hamming distance between the N matching average hash values ​​and the matching template average hash value. The Hamming distance between the i-th matching average hash value and the matching template average hash value is denoted as d i ,

[0163] S3.3.2: Determine the template image corresponding to the minimum Hamming distance, and use the template image corresponding to the minimum Hamming distance as the matching result image of template matching; the calculation method of the Hamming distance is well known to those skilled in the art;

[0164] The present invention performs matching test by using the absolute error sum algorithm, the error square sum algorithm, the normalized product correlation algorithm, the zero mean normalized cross-correlation algorithm and the method proposed in this article, and the matching operation time is shown in the table.

[0165]

[0166] The specific process of obtaining the sub-region to be detected of the image to be detected according to the template matching result picture in S3.4 is as follows: using affine transformation to map the detection area obtained in S2.1.5 to the matching result picture of template matching, to obtain the sub-region to be detected of the image to be detected;

[0167] The affine transformation formula of an image is:

[0168] f(X)=AX+b

[0169] X represents the pixel matrix of the detection area, f(X) represents the pixel matrix mapped to the template matching result image after affine transformation, A represents the transformation matrix of affine transformation, and the transformation matrix formula of affine transformation is:

[0170]

[0171] R 00 , R01 , R 10 and R 11 Both represent rotation variables, T x represents the lateral translation variable, T y represents the longitudinal translation variable;

[0172] When the detection image is collected, the camera height and the base are fixed, so the collected image may only be translated or rotated. When the constraints are determined, R can be determined. 00 , R 01 , R 10 and R 11 The values ​​of are -cosθ, sinθ, -sinθ and cosθ, where θ represents the rotation angle.

[0173] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0174] Specific implementation method 9: This implementation method is different from specific implementation methods 1 to 8 in that:

[0175] In S4, defect detection is performed on the sub-region to be detected in the image to be detected, a detection result is obtained, and the detection result is input visually using visualization software. The specific process is as follows:

[0176] S4.1: Acquire characteristic parameters of colloids in historical detection images; extract characteristic parameters of colloids in the sub-region to be detected in the image to be detected;

[0177] The characteristic parameters of the colloid include: colloid perimeter, colloid area, colloid circularity, colloid rectangularity, colloid perimeter ratio, and colloid center coordinates;

[0178] The area size is expressed by the formula: Where f(i,j) represents the target area, and m and n represent the number of pixel horizontal and vertical coordinates in the area, respectively.

[0179]

[0180] Then the perimeter can be expressed as a formula.

[0181]

[0182] The coordinates of the center point are obtained by adding the horizontal coordinates of all pixels in the target area and dividing them by the area, and by adding the vertical coordinates of all pixels and dividing them by the area, as shown in the formula.

[0183]

[0184] The circularity is measured by comparing the area and the square of the perimeter and then multiplying it by 4π. The result is the circularity of the target area, where the circularity ranges from (1, +∞), as shown in the formula.

[0185]

[0186] Rectangularity reflects the approximation of the object's outline, that is, the similarity between the object's outline and a rectangular outline. Its value range is usually between 0 and 1. The closer it is to a rectangle, the closer the value is to 1.

[0187]

[0188] Where a and b represent the length and width of the minimum bounding rectangle of the target area, respectively.

[0189] The perimeter ratio is the ratio of the perimeter of the target area to the longer side of the circumscribed rectangle of the target area.

[0190] S4.2: performing defect detection on the sub-region to be detected in the image to be detected according to the characteristic parameters of the colloid in the historical detection image and the characteristic parameters of the colloid in the sub-region to be detected in the image to be detected, and obtaining a detection result;

[0191] S4.3: Use Qt Creator software to visualize the test results;

[0192] The visualization interface of the present invention is designed using Qt Creator open source software. Qt provides a wealth of GUI components that can effectively build interfaces and visual windows. It is also bound to multiple languages ​​to make its functions complete. It also has rich image processing functions and supports image loading and transformation, which is suitable for presenting the test results. This improves the efficiency of detection.

[0193] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0194] Specific implementation method 10: This implementation method is different from specific implementation methods 1 to 9 in that:

[0195] In the S4.2, according to the characteristic parameters of the colloid in the historical detection image and the characteristic parameters of the colloid in the sub-region to be detected in the image to be detected, the defect detection is performed on the sub-region to be detected in the image to be detected, and the detection result is obtained; the specific process is:

[0196] S4.2.1: Determine whether the sub-area to be inspected in the image to be inspected has a defect state of colloid offset by the colloid center coordinates. The specific process is as follows:

[0197] According to the colloid center coordinates of the historical detection image, set the center coordinate interval.

[0198] When the center coordinates of the colloid in the sub-area to be detected in the image to be detected exceed the center coordinate interval, it is judged as a defect state of colloid offset;

[0199] When the center coordinates of the colloid in the sub-area of ​​the image to be tested are within the normal range, it is judged that there is no colloid offset defect; if the colloid is offset, the center coordinate will be greatly offset. Considering the possibility of glue breakage, a two-dimensional value range can be set by detecting the distance between the horizontal and vertical coordinates of the center coordinate and the normal position. If any coordinate exceeds this range, it can be judged as a colloid offset defect.

[0200] S4.2.2: Determine whether the sub-area to be inspected in the image to be inspected has a defect state of broken glue according to the rectangularity; the specific process is as follows:

[0201] According to the colloid rectangularity of the characteristic parameters of the historical detection image, a colloid rectangularity threshold is set;

[0202] When the colloid rectangularity of the sub-area to be detected in the image to be detected is less than the colloid rectangularity threshold, it is determined that there is a defect state of colloid breakage;

[0203] When the colloid rectangularity of the sub-area to be detected in the image to be detected is not less than the colloid rectangularity threshold, it is determined that there is no defect state of colloid breakage;

[0204] After eliminating the influence of colloid offset, the glue dispensing area of ​​the electronic component is approximately rectangular, and in the absence of glue breakage, its rectangularity should be close to 1. The experimental results show that the rectangularity of the glue dispensing area of ​​the glue breakage electronic component is significantly less than 1. Therefore, a reasonable rectangularity range can be set. When the calculated rectangularity is not within this range, it can be judged that there is a glue breakage problem.

[0205] S4.2.3: Binarize the sub-region to be inspected in the image to be inspected, and determine whether there is a defect state of insufficient glue or no glue based on the image to be inspected after the binarization process. The specific process is as follows:

[0206] S4.2.3.1: Set a no-glue threshold, and set the pixel values ​​in the sub-region to be detected in the image to be detected that are higher than the no-glue threshold to 255; obtain the image to be detected after no-glue binarization processing;

[0207] According to the image to be detected after the no-glue binary processing and the image of the electronic component to be detected without the no-glue binary processing, it is judged whether the sub-area to be detected in the image to be detected has a no-glue defect state.

[0208] When the white area does not change, it is judged as a defective state of glue;

[0209] When the white area changes, it is judged that there is no defective state of no glue;

[0210] The pixel value of white is 255;

[0211] S4.2.3.2: Set a low-glue threshold, and set the pixel values ​​in the sub-region to be detected in the image to be detected that are higher than the low-glue threshold to 255; obtain the image to be detected after low-glue binarization processing;

[0212] According to the image to be detected after the low-resin binary processing and the image of the electronic component to be detected without the low-resin binary processing, it is determined whether the sub-area to be detected in the image to be detected has a low-resin defect state.

[0213] When the number of newly added white areas is less than 1, it is judged that there is no defect state of insufficient glue;

[0214] When the number of newly added white areas is not less than 1, it is judged as a defective state with insufficient glue.

[0215] In order to distinguish the electronic components in the three glue dispensing states of normal, no glue and little glue, the color characteristics of the glue dispensing area can be processed. The electronic components with normal glue dispensing are gray, and the glue dispensing area without glue is white. Through binarization processing, these three states can be effectively distinguished. In the state of no glue, the glue dispensing area is close to white. By setting a threshold for binarization, the glue-free electronic components can be accurately identified. The remaining images include electronic components in normal and little glue states, which are further distinguished by detecting the little glue area. If a white no glue area appears in the little glue area, it is determined to be a problem of little glue. Finally, this method can successfully classify and identify normal glue dispensing samples.

[0216] Fig.11 The a in the figure shows the status of reading the matching image and reading the template. Fig.11 Figures bf show the detection results for different defect images. In the experiment, 100 samples were extracted for each type of dispensing electronic components, totaling 500 samples. Due to the small number of colloid offset samples, noise was added to expand the offset sample set, and 10 experiments were conducted, and the average value was finally taken as the result. Through the improved template matching method, the system can accurately locate and detect different types of dispensing defects. The experimental results demonstrate the effectiveness and reliability of the detection system in industrial applications, and provide a feasible solution for automatic detection of dispensing defects. The other steps and parameters are the same as those of Specific Implementation Methods 1 to 9.

[0217] In this experiment, 100 samples were extracted for each type of dispensing electronic components, totaling 500 samples. Since there were too few samples of colloid offset, considering adding noise to the dispensing electronic components with existing colloid offset defects, the noise-contaminated samples were added to the collection of colloid offset electronic components, and each type of dispensing electronic components was tested 10 times, and the average of the results of multiple experiments was taken to obtain the final experimental data. The results are shown in the table.

[0218]

[0219] The detection method of the present invention performs well in the detection of defective dispensing electronic components such as no glue, broken glue and glue deviation, and the accuracy rate reaches 100%, which verifies the effectiveness of the present invention.

[0220] The above only describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above specific implementation methods. Although the present invention has been disclosed as above in the preferred embodiments, it is not used to limit the present invention. Any technician familiar with the profession can make some changes or modify the technical contents disclosed above into equivalent embodiments with equivalent changes without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A defect detection method based on Zernike orthogonal moments and hash algorithm, characterized in that: The following steps are involved: S1: Collecting images to be detected; the images to be detected include images of original parts without glue dots and images of original parts with glue dots; S2: Generate a matching image and a matching template according to the image to be detected; S3: performing template matching according to the matching image and the matching template to obtain a sub-region to be detected of the image to be detected; S4: Perform defect detection on the sub-region to be detected of the image to be detected, obtain the detection result, and visualize the detection result using visualization software.

2. The defect detection method based on Zernike orthogonal moments and hash algorithm according to claim 1, characterized in that: The camera for collecting the image to be detected in S1 is a CMOS camera; the lighting method is front illumination, and the light source is an LED ring light source.

3. The defect detection method based on Zernike orthogonal moments and hash algorithm according to claim 2, characterized in that: In S2, a matching image is generated according to the image to be detected; the specific process is as follows: S2.1: obtaining an image of the original part without glue spots after extracting the area to be inspected according to the image of the original part without glue spots; Obtaining an image of the dispensing original part after extracting the area to be inspected according to the dispensing original part image; S2.2: extracting the original image of the non-dot glue after the area to be detected is used as a matching template; Extract the image of the dispensing original after the area to be inspected to obtain a matching image; In the step S2.1, the image of the original part without dots of glue is obtained after the area to be inspected is extracted according to the image of the original part without dots of glue; The specific process of obtaining the dispensing original image after extracting the area to be detected based on the dispensing original image is as follows: S2.1.1: performing bilateral filtering on the image of the original part without dots of glue to obtain the original image of the original part without dots of glue after bilateral filtering; Performing bilateral filtering on the dispensing original image to obtain the dispensing original image after bilateral filtering; S2.1.2: performing image enhancement processing on the image of the original part without dots of glue after bilateral filtering to obtain an enhanced image of the original part without dots of glue; Performing image enhancement processing on the dispensing original image after bilateral filtering to obtain an enhanced dispensing original image; S2.1.3: performing image threshold segmentation processing on the enhanced non-dot glue original image to obtain the non-dot glue original image after image threshold segmentation processing; Performing image threshold segmentation processing on the enhanced dispensing original image to obtain the dispensing original image after image threshold segmentation processing; S2.1.4: extracting the sub-pixel edge contour of the non-dot glue original image after the image threshold segmentation processing; obtaining the sub-pixel edge contour of the non-dot glue original image; Extracting sub-pixel edge contours of the dispensing original image after the image threshold segmentation process; obtaining the sub-pixel edge contours of the dispensing original image; S2.1.5: Obtain the area to be detected according to the sub-pixel edge contour of the original image without glue dot and the sub-pixel edge contour of the original image with glue dot; Extracting the enhanced non-dot glue original image according to the area to be detected, and obtaining the non-dot glue original image after extracting the area to be detected; The enhanced dispensing original image is extracted according to the area to be detected, so as to obtain the dispensing original image after the area to be detected is extracted.

4. The defect detection method based on Zernike orthogonal moments and hash algorithm according to claim 3 is characterized in that: In the above 2.1.2, the image enhancement processing is performed on the non-dot glue original image after bilateral filtering to obtain the enhanced non-dot glue original image; the image enhancement processing is performed on the dot glue original image after bilateral filtering to obtain the enhanced dot glue original image; the specific process is: S2.1.2.1: Calculate the membership degree of each pixel of the bilaterally filtered point-free glue original image; Calculate the membership degree of each pixel of the bilaterally filtered dispensing original image; S2.1.2.2: Perform data enhancement processing on each pixel of the dot-free glue original image according to the membership degree of each pixel of the dot-free glue original image to obtain an enhanced dot-free glue original image pixel, and all enhanced dot-free glue original image pixels constitute an enhanced dot-free glue original image; Data enhancement processing is performed on each pixel of the glue dispensing original image according to the obtained membership degree of each pixel of the glue dispensing original image to obtain enhanced glue dispensing original image pixels, and all enhanced glue dispensing original image pixels constitute the enhanced glue dispensing original image.

5. The defect detection method based on Zernike orthogonal moments and hash algorithm according to claim 4, characterized in that: The specific process of performing image threshold segmentation processing on the image in S2.1.3 to obtain the image after image threshold segmentation processing is as follows: S2.1.3.1: Set the global threshold T, Set the pixel value to 255 at the position where the pixel value in the enhanced image is higher than the global threshold T; Set the pixel value of the enhanced image to 0 if the pixel value is not higher than the global threshold T; Finally, the image after preliminary segmentation processing is obtained; S2.1.3.2: Set the area threshold of the connected area. Calculate the area of ​​the connected region in the image after the initial segmentation process. The connected area refers to an area with the same pixel value; For connected areas whose area is smaller than the connected area threshold, the pixel value is set to 255; Finally, the image after image threshold segmentation is obtained; In the step S2.1.4, sub-pixel edge contour extraction is performed on the dot-free glue original image after the image threshold segmentation process; and the dot-free glue original image after the sub-pixel edge contour extraction is obtained; Sub-pixel edge contour extraction is performed on the dispensing original image after the image threshold segmentation processing; the dispensing original image after sub-pixel edge contour extraction is obtained, and the specific process is as follows: The sub-pixel edge contour of the non-dot glue original image after image threshold segmentation is extracted by using the Zernike orthogonal moment method to obtain the sub-pixel edge contour of the non-dot glue original image. The sub-pixel edge contour of the dispensing original image after image threshold segmentation is extracted by using the Zernike orthogonal moment method to obtain the sub-pixel edge contour of the dispensing original image. The area to be inspected in S2.1.5 includes an overflow inspection area and a insufficient inspection area; In the S2.1.5, the area to be detected is obtained according to the sub-pixel edge contour of the original image without glue dots and the sub-pixel edge contour of the original image with glue dots. The specific process is as follows: S2.1.5.1: Fill the sub-pixel edge contour of the original image without glue to obtain the original glue-free area; Fill the sub-pixel edge contour of the original glue-dispensing image to obtain the original glue-containing area; S2.1.5.2: Perform expansion morphology treatment on the original rubber-containing area to obtain an expansion morphology-treated rubber-containing area; Performing corrosion morphology treatment on the original glue-containing area to obtain a corrosion morphology treated glue-containing area; S2.1.5.3: Perform differential processing on the original glue-free area and the original glue-dispensing part image after expansion morphology processing to obtain the glue overflow detection area; The glue-less detection area is obtained by performing differential processing on the original glue-free area and the glue-dispensing original image after corrosion morphology processing.

6. The defect detection method based on Zernike orthogonal moments and hash algorithm according to claim 5, characterized in that: The specific process of calculating the membership degree of a certain pixel of the image in S2.1.2.1 is as follows: Where x represents the pixel value, x1 represents the first pixel threshold, x2 represents the second pixel threshold, x3 represents the third pixel threshold, μ d (x) represents the dark pixel membership, μ g (x) represents the gray pixel membership, μ b (x) represents the bright pixel membership; The membership degree of a certain pixel obtained in S2.1.2.2 is used to perform data enhancement processing on a certain pixel; the specific process is: where s d Indicates the value of a completely black pixel, s g Represents the grayscale pixel value, s b Represents the all-white pixel value, and s represents the pixel value at the current position after data enhancement processing.

7. The defect detection method based on Zernike orthogonal moments and hash algorithm according to claim 6, characterized in that: In S3, template matching is performed according to the matching image and the matching template to obtain the sub-region to be detected of the image to be detected. The specific process is as follows: S3.1: Determine a minimum matching image size, and divide the matching image into N matching images according to the minimum matching image size, where N is a positive integer; S3.2: Calculate the average hash value of N matching images to obtain N matching average hash values; Calculate the average hash value of the matching template; Get the average hash value of the matching template; S3.3: Perform template matching based on the N matching average hash values ​​and the matching template average hash value to obtain a template matching result image. S3.4: Obtain the sub-region to be detected of the image to be detected according to the template matching result image.

8. The defect detection method based on Zernike orthogonal moments and hash algorithm according to claim 7, characterized in that: In S3.3, template matching is performed according to the N matching average hash values ​​and the matching template average hash value to obtain a matching result image of the template matching. The specific process is as follows: S3.3.1: Calculate the Hamming distance between the N matching average hash values ​​and the matching template average hash value. The Hamming distance between the i-th matching average hash value and the matching template average hash value is denoted as d i , S3.3.2: Determine the template image corresponding to the minimum Hamming distance, and use the template image corresponding to the minimum Hamming distance as the matching result image of the template matching; The specific process of obtaining the sub-region to be detected of the image to be detected according to the template matching result image in S3.4 is as follows: the sub-region to be detected obtained in S2.1.5 is mapped to the matching result image of the template matching by using affine transformation to obtain the sub-region to be detected of the image to be detected; it can be expressed as: f(X)=AX+b X represents the pixel matrix of the area to be detected, f(X) represents the pixel matrix mapped to the template matching result image after affine transformation, A represents the transformation matrix of affine transformation, and the transformation matrix formula of affine transformation is: R 00 , R 01 , R 10 and R 11 Both represent rotation variables, T x represents the lateral translation variable, T y Represents the vertical translation variable.

9. The defect detection method based on Zernike orthogonal moments and hash algorithm according to claim 6, characterized in that: The specific process of performing defect detection on the sub-region to be detected in the image to be detected in S4, obtaining the detection result, and visually outputting the detection result using visualization software is as follows: S4.1: Acquire characteristic parameters of colloids in historical detection images; extract characteristic parameters of colloids in the sub-region to be detected in the image to be detected; The characteristic parameters of the colloid include: colloid rectangularity, colloid center coordinates; S4.2: performing defect detection on the sub-region to be detected in the image to be detected according to the characteristic parameters of the colloid in the historical detection image and the characteristic parameters of the colloid in the sub-region to be detected in the image to be detected, and obtaining a detection result; S4.3: Use Qt Creator software to visualize the detection results.

10. The defect detection method based on Zernike orthogonal moments and hash algorithm according to claim 9, characterized in that: In the S4.2, according to the characteristic parameters of the colloid in the historical detection image and the characteristic parameters of the colloid in the sub-region to be detected in the image to be detected, the defect detection is performed on the sub-region to be detected in the image to be detected, and the detection result is obtained; the specific process is: S4.2.1: Determine whether the sub-area to be inspected in the image to be inspected has a defect state of colloid offset by the colloid center coordinates. The specific process is as follows: According to the colloid center coordinates of the historical detection image, set the center coordinate interval. When the center coordinates of the colloid in the sub-area to be detected in the image to be detected exceed the center coordinate interval, it is judged as a defect state of colloid offset; When the center coordinates of the colloid in the sub-area to be detected in the image to be detected do not exceed the center coordinate interval, it is determined that there is no defect state of colloid deviation; S4.2.2: Determine whether the sub-area to be inspected in the image to be inspected has a defect state of broken glue according to the rectangularity; the specific process is as follows: According to the colloid rectangularity of the characteristic parameters of the historical detection image, a colloid rectangularity threshold is set; When the colloid rectangularity of the sub-area to be detected in the image to be detected is less than the colloid rectangularity threshold, it is determined that there is a defect state of colloid breakage; When the colloid rectangularity of the sub-area to be detected in the image to be detected is not less than the colloid rectangularity threshold, it is determined that there is no defect state of colloid breakage; S4.2.3: Binarize the sub-region to be detected in the image to be detected, and determine whether there is a defect state of insufficient glue or no glue based on the image to be detected after the binarization process. The specific process is as follows: S4.2.3.1: Set a no-glue threshold, and set the pixel values ​​in the sub-region to be detected in the image to be detected that are higher than the no-glue threshold to 255; obtain the image to be detected after no-glue binarization processing; According to the image to be detected after the no-glue binary processing and the image to be detected without the no-glue binary processing, it is judged whether the sub-area to be detected of the image to be detected has a no-glue defect state, When the white area does not change, it is judged as a defective state of glue; When the white area changes, it is judged that there is no defective state of no glue; The pixel value of white is 255; S4.2.3.2: Set a low-glue threshold, and set the pixel values ​​in the sub-region to be detected in the image to be detected that are higher than the low-glue threshold to 255; obtain the image to be detected after low-glue binarization processing; According to the image to be detected after the low-glue binary processing and the image to be detected without the low-glue binary processing, it is judged whether the sub-area to be detected in the image to be detected has the defect state of low-glue. When the number of newly added white areas is less than 1, it is judged that there is no defect state of insufficient glue; When the number of newly added white areas is not less than 1, it is judged as a defective state of insufficient glue.

Images