A method for counting electronic components based on physical size matching

Abstract

The invention discloses an electronic component counting method based on physical size matching, and relates to the technical field of electronic component counting. The invention uses a SURF algorithm to count materials, adopts a binocular camera to shoot materials, uses a SURF algorithm to splice material images, extracts the contour of the material, calculates the size of the material and saves it; the binocular camera shoots a different number of materials from the first time, uses a SURF algorithm to splice the images of the materials, extracts the contour and segments the images, calculates the size of the materials again, matches the material size saved for the first time with it to obtain the number of materials, and uses a gray model to judge whether the result obtained based on the physical size matching method is stable. If the result is unstable, the material size of the different number of materials for the first time is re-shaken and then the material size is calculated by the SURF algorithm and matched with the material size saved for the first time. If the result is stable, the size is matched until the number is stable and the repeated matching is stopped.

Description

Technical Field

[0001] The invention relates to the technical field of electronic component counting, and in particular to an electronic component counting method based on physical size matching. Background Art

[0002] The present invention relates to an electronic component counting method, which is used for counting electronic components of various shapes, and the counting method is of great significance in the production and detection of electronic components. Traditional manual counting not only requires a large amount of labor costs, but also requires the counting personnel to have high familiarity with electronic components and counting familiarity, but there are still problems such as low counting efficiency and low counting accuracy. The electronic component counting method based on template matching searches for a target in a large image. The target needs to have the same size, direction and image elements as the template. If the matching target in the original image is rotated, inverted or the size changes, the algorithm is invalid, so that the electronic components cannot be counted normally. Summary of the invention

[0003] The purpose of the present invention is to provide an electronic component counting method based on physical size matching to solve the problems proposed in the above background technology that manual counting requires a lot of labor costs and is inefficient and prone to errors, and that the matching target in the template matching technical method can only move in parallel, otherwise the algorithm is invalid.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: an electronic component counting method based on physical size matching, the method comprising two links: a learning link and a counting link.

[0005] The learning process includes the following steps:

[0006] Step 1: illuminate a light-transmitting empty tray with a uniform light source, use a dual-camera device to capture images, stitch the images, and store the images as background image 1;

[0007] Step 2: After placing a quantity of materials ξ on an empty tray that is transparent to light, a dual-camera device is used to capture images, stitch the images, and store the images as image 2. Image 2 minus background image 1 generates foreground image 3;

[0008] Step 3: Model the three postures of the material in the foreground image 3 and perform contour extraction, calculate the length, width, height and area physical size information of the material, and store the information in a database file. The three postures include a horizontal posture, a side posture and a vertical posture;

[0009] The counting process comprises the following steps:

[0010] Step 4: Place a quantity of β new materials on the tray, β>ξ; after the materials are shaken apart, use a dual-camera device to capture images, stitch the images, store the images as image 4, and subtract background image 1 from image 4 to generate foreground image 5; extract the image contour based on foreground image 5, and calculate the length, width, height and area physical dimensions of the materials;

[0011] Step 5: The physical dimensions of length, width, height and area of ​​the materials calculated in step 4 are matched with the information in the database file described in step 3. The number of individual materials is first counted, and then the distance transformation and watershed segmentation methods are used to segment multiple materials that are close together or stacked together, and then the numbers are counted. For materials that are close together or stacked together and cannot be divided, the number of materials is counted according to the area method;

[0012] Step 6: Use the grey model to determine whether the result obtained based on the physical size matching method is stable; if the result obtained based on the physical size matching method is unstable, repeat step 4 to recalculate the new length, width, height and area physical dimensions of the material;

[0013] Step 7: Repeat step 5 until the number of material counting results that appear the same reaches a threshold value M, which is recorded as a counting stability situation, and the material counting results corresponding to the same number of times appear are taken as the final results.

[0014] The dual-camera device collects images and stitches the images, and uses the SURF algorithm to extract and match features to generate images with higher resolution;

[0015] The specific steps are as follows: S2-1. Use the SURF algorithm to extract feature points in different material images; S2-2. Use the SURF algorithm to extract candidate feature points in the scale control, and then use the three-dimensional linear interpolation method to locate the candidate points; S2-3. Determine the main direction of the feature points in different material images; S2-4. Construct the corresponding SURF feature descriptor based on the feature points.

[0016] The SURF algorithm is used to extract feature points in different material images;

[0017] The specific method is as follows: The SURF algorithm uses the Hessian matrix determinant to detect feature points. First, the image is convolved with a Gaussian template, and then a Hessian matrix with scale information is obtained at the pixel point. The expression is:

[0018]

[0019] Where L(x,y,σ)=G(x,y,σ)*I(x,y), G(x,y,σ) is a scale-variable Gaussian function, I(x,y) is the image, Lxx (x,y,σ),L xy (x,y,σ),L yy (x, y, σ) are the second-order differentials in the x, xy, and y directions at the pixel point (x, y), respectively; therefore, the Hessian matrix determinant is ultimately simplified to:

[0020] det(H)=L xx L yy -L xy L xy

[0021] The approximate discriminant of the Hessian matrix for each pixel is:

[0022] det(H)=L xx L yy -(ωL xy ) 2

[0023] ω represents the weighting coefficient, and det(H) represents the box filter response value in the area near the point.

[0024] The SURF algorithm is used to extract candidate feature points in the scale control;

[0025] The specific method is as follows: In order to obtain sampling points of different scales, it is necessary to construct the scale space of the image and extract feature points in the scale space. The scale space of SURF is composed of O groups and L layers. Different layers in the same group use filters of the same size, but the blur coefficient of the filter gradually increases; according to the pyramid structure of the SURF algorithm, it keeps the original image unchanged and only changes the filter size;

[0026] The positioning of SURF feature points is to use neighborhood non-maximum suppression on the response images of feature points of different scales. In the constructed SURF pyramid, each pixel point passing through the Hessian matrix is ​​compared with 26 points in the two-dimensional image space and scale space neighborhood to select candidate feature points; then the three-dimensional linear interpolation method is used to locate the candidate points to obtain sub-pixel level feature points, thereby completing the extraction of feature points.

[0027] Determining the main directions of feature points in different material images;

[0028] In order to meet the rotation invariance, the main direction of the feature point must be determined. The specific method is: count the Haar wavelet features in the feature point area, and in the feature point area, count the horizontal Haar wavelet features and vertical Haar wavelet features of all points in a 60-degree sector, and get a response value for each sector; add up the response values ​​to form a vector, and select the longest vector direction as the main direction of the final feature point.

[0029] The corresponding SURF feature descriptor is constructed according to the feature points;

[0030] The specific method is as follows: take a 4×4 rectangular area block around the feature point, and the direction of the obtained rectangular area is along the main direction of the feature point; divide the rectangular area block into 16 equal sub-areas, and count the Haar wavelet features of 25 pixels in the horizontal and vertical directions in each sub-area; the horizontal and vertical directions mentioned above are relative to the main direction;

[0031] The Haar wavelet features are: the sum of horizontal values ​​∑dx, the sum of vertical values ​​∑dy, the sum of absolute values ​​in the horizontal direction |Σdx|, and the sum of absolute values ​​in the vertical direction |Σdy|; the Haar wavelet value is used as the feature vector of each sub-block area, and a total of 64-dimensional vectors are used as the descriptors of the SURF features.

[0032] After the image of the material is collected, the image contour is extracted and the length, width and height of the material are calculated. The specific method is: in the learning stage, image 2 and background image 1 are used for difference, and in the counting stage, image 4 and background image 1 are used for difference to obtain a binary image, and then the binary image is subjected to a morphological opening operation to remove interference from component pins, and the opening operation adopts a cross convolution kernel;

[0033] The size of the cross convolution kernel is determined by the following method steps: S7-1, the circumscribed rectangle of the material is subtracted from the inscribed rectangle of the material to obtain the interference image of the material pins; S7-2, the minimum convolution kernel size d is used to erode the interference image by morphological corrosion operation; S7-3, the actual convolution kernel size is determined to be Len = 1.5×d; then the image contour is extracted, and the length l, width w, height h and area a of the material are counted.

[0034] The physical dimensions of the material, such as length, width, height and area, are matched with the information in the database file, and then the number of materials is counted according to the area method; the physical dimension matching conditions of the material are as follows:

[0035] S8-1. Horizontal placement judgment conditions:

[0036] L×0.8≤l≤L×1.2∧W×0.8≤w≤W×1.2∧A 平放 ×0.8≤a≤A 平放 ×1.2

[0037] S8-2, side placement judgment conditions:

[0038] L×0.8≤l≤L×1.2∧H×0.8≤h≤H×1.2∧A 侧放 ×0.8≤a≤A 侧放 ×1.2

[0039] S8-3, vertical placement judgment conditions:

[0040] W×0.8≤w≤W×1.2∧H×0.8≤h≤H×1.2∧A 竖放 ×0.8≤a≤A 竖放 ×1.2

[0041] Among them, L, W, H, A 平放 , A 侧放 , A 竖放 They respectively represent the length, width, height, horizontal area, side area and vertical area information of the file created in the learning phase; l, w, h and a respectively represent the length, width, height and area information of the material;

[0042] For materials that do not meet the above conditions and have two or more adhesion superpositions, distance transformation and watershed segmentation methods are used; distance transformation is that the gray value of each pixel in the binary image is the distance between the pixel and the nearest background pixel; the measurement metric used in this method is Euclidean distance; the main process of distance transformation is as follows:

[0043] Assuming a binary image, containing a connected region S, in which there are target set O and background set B, the distance transform is defined as follows:

[0044] D(p)=Min(disf(p,q)) p∈O,q∈B

[0045] Where D(p) represents the distance transformation value of point p; p represents a point in the target set O; q represents a point in the background set B;

[0046] S8-4, classifying target pixels in the image into internal points, external points and isolated points;

[0047] Taking the four-domain area of ​​the central pixel as an example, if the central pixel is the target pixel (value is 1) and the four neighbors are all target pixels (value is 1), then the point is an internal point. If the central pixel is the target pixel and the four neighbors are background pixels (value is 0), then the central point is an isolated point, as shown in the figure below. The target area points other than the internal points and isolated points are boundary points;

[0048] S8-5, calculate all the internal points and non-interior points in the image, the point sets are S1 and S2 respectively;

[0049] S8-6. For each internal point (x, y) in S1, use the distance formula disf() to calculate the minimum distance in S2. All minimum distances constitute a set S3.

[0050] S8-7, calculate the maximum value Max and the minimum value Min in S3;

[0051] S8-8. For each internal point, the calculation formula of the converted grayscale value G(x,y) is as follows:

[0052] G(x,y)=255×|S3(x,y)-Min| / |Max-Min|

[0053] Where S3(x,y) represents the shortest distance between the point (x,y) in S1 and S2;

[0054] S8-9, remain unchanged for isolated points;

[0055] In the distance transformation, the distance function disf() is selected as the Euclidean distance:

[0056]

[0057] Where x and y represent the coordinates of a point in the plane, p(x1,y1) and q(x2,y2) represent the coordinates of p and q respectively;

[0058] For materials that are stuck together and cannot be separated, first use the area method to count the number of them. The formula is as follows:

[0059]

[0060] Where n represents the number of statistics, a represents the total area of ​​the materials superimposed together, and A represents the area of ​​one material study;

[0061] When n=1 and l>L×1.2∨w>W×1.2, then n=2.

[0062] The specific steps of using the grey model to judge the stability of the results obtained based on the physical size matching method are as follows:

[0063] S9-1, Accumulation Generation

[0064] x (0) ={x (0) (1),x (0) (2),x (0) (3),…,x (0) (N)}

[0065] x (0) The result obtained from the size matching from step 4 to step 5;

[0066] x (0) (1) is the result of the first match from step 4 to step 5;

[0067] x (0) (2) is the result obtained from the second matching from step 4 to step 5;

[0068] x(0) (3) is the result obtained from the third matching from step 4 to step 5;

[0069] x (0) (N) is the result obtained from the Nth matching from step 4 to step 5;

[0070] Let x 1 (1) = x (0) (1)

[0071] x 1 (2) = x (0) (1)+x (0) (2)

[0072] x 1 (3) = x (0) (1)+x (0) (2)+x (0) (3)

[0073] So we get a new data sequence:

[0074] x 1 ={x 1 (1),x 1 (2),x 1 (3),…,x 1 (N)}

[0075] S9-2. Establishing the GM(1,1) model

[0076] x (0) (N)+kz 1 (N) = b is the basic form of the GM (1,1) model (N = 2, 3, ..., n), where k is called the development coefficient, z 1 (N) is called the whitening background value, b is called the gray action amount; introducing the matrix formula, we can get:

[0077]

[0078] So, the GM(1,1) model x (0) (N)+kz 1 (N) = b can be expressed as Y = Bu, using the normal equation (B T B) -1 B T Y uses the principle of least squares to find the values ​​of k and b;

[0079] S9-3. Prediction

[0080] The corresponding whitening model is From this we can get

[0081] Let Q = N + 1 to get the predicted value

[0082] Where N = 0, 1, 2, 3, ..., n-1; Q = N+1; Represents the prediction of the n-time cumulative sum;

[0083] S9-4. Inspection

[0084] The predicted value Generate using the cumulative subtraction method

[0085] Where N = 0, 1, 2, 3, ..., n-1; Q = N+1;

[0086] Represents the predicted value when the number of matches is n;

[0087] Calculate residuals

[0088] Where N = 0, 1, 2, 3, ..., n-1; Q = N+1;

[0089] Calculate the mean of the residuals

[0090]

[0091] Calculate x (0) Mean of (N)

[0092]

[0093] Calculate the original sequence x (0) The variance of (N) is Y1 and the variance of the residual e(N) is Y2

[0094]

[0095] Calculate the posterior difference ratio

[0096]

[0097] Calculating the probability of small errors

[0098]

[0099] When C<0.35 and P>0.95, the grey model has a good accuracy level;

[0100] When C<0.45 and P>0.8, the accuracy level of the grey model is qualified;

[0101] When C<0.5 and P>0.7, the accuracy level of the grey model is marginal;

[0102] When C≥0.65 and P≤0.7, the accuracy level of the grey model is unqualified;

[0103] When the grey model accuracy level is good after inspection, it means that the result obtained by the physical size matching method is stable.

[0104] Compared with the prior art, the beneficial effects achieved by the present invention are: the present invention can realize rapid and correct counting of large quantities of electronic components, reduce labor costs, and avoid unnecessary waste of resources. Compared with the electronic component counting method based on template matching, the present invention allows partial superposition of materials, automatically removes interference from electronic component pins, can separate materials that are partially close together, and has better robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0106] Figure 1 The present invention is a flowchart of an electronic component counting method based on physical size matching for electronic component counting. DETAILED DESCRIPTION

[0107] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0108] See also Figure 1 ,This implementation case provides an electronic component counting method based on physical size matching, which includes two steps: a learning step and a counting step.

[0109] The learning process includes the following steps:

[0110] Step 1: illuminate a light-transmitting empty tray with a uniform light source, use a dual-camera device to capture images, stitch the images, and store the images as background image 1;

[0111] Step 2: After placing a quantity of materials ξ on a light-permeable empty tray, a dual-camera device is used to capture images, stitch the images, and store the images as image 2. Image 2 is subtracted from background image 1 to generate foreground image 3;

[0112] Step 3: Model the three postures of the material in the foreground image 3 and extract the contours, calculate the length, width, height and area physical size information of the material, and store the information in a database file. The three postures include horizontal posture, side posture and vertical posture;

[0113] The counting process comprises the following steps:

[0114] Step 4: Place a quantity of β new materials on the tray, β>ξ; after the materials are shaken apart, use a dual-camera device to capture images, stitch the images, store the images as image 4, and subtract background image 1 from image 4 to generate foreground image 5; extract the image contour based on foreground image 5, and calculate the length, width, height and area physical dimensions of the materials;

[0115] Step 5: The physical dimensions of length, width, height and area of ​​the materials calculated in step 4 are matched with the information in the database file described in step 3. The number of individual materials is first counted, and then the distance transformation and watershed segmentation methods are used to segment multiple materials that are close together or stacked together, and then the numbers are counted. For materials that are close together or stacked together and cannot be divided, the number of materials is counted according to the area method;

[0116] Step 6: Use the grey model to determine whether the result obtained based on the physical size matching method is stable; if the result obtained based on the physical size matching method is unstable, repeat step 4 to recalculate the new length, width, height and area physical dimensions of the material;

[0117] Step 7: Repeat step 5 until the number of material counting results that appear the same reaches the threshold value M, which is recorded as a stable counting situation, and the material counting results corresponding to the number of times that appear the same are taken as the final results;

[0118] The dual-camera device collects images and stitches the images, and uses the SURF algorithm to extract and match features to generate images with higher resolution;

[0119] The specific steps are as follows: S2-1. Use the SURF algorithm to extract feature points in different material images; S2-2. Use the SURF algorithm to extract candidate feature points in the scale control, and then use the three-dimensional linear interpolation method to locate the candidate points; S2-3. Determine the main direction of the feature points in different material images; S2-4. Construct the corresponding SURF feature descriptor based on the feature points.

[0120] The specific method of extracting feature points in different material images using the SURF algorithm is as follows:

[0121] The SURF algorithm uses the Hessian matrix determinant to detect feature points. First, the image is convolved with a Gaussian template, and then a Hessian matrix with scale information is obtained at the pixel point. The expression is:

[0122]

[0123] Where L(x,y,σ)=G(x,y,σ)*I(x,y), G(x,y,σ) is a scale-variable Gaussian function, I(x,y) is the image, L xx (x,y,σ),L xy (x,y,σ),L yy (x, y, σ) are the second-order differentials in the x, xy, and y directions at the pixel point (x, y). Therefore, the Hessian matrix determinant is finally simplified to:

[0124] det(H)=L xx L yy -L xy L xy

[0125] Among them, det(H) is used to detect extreme points. In order to balance the error caused by using the box filter approximation, D xy Multiply it by ω, ω is 0.9. The approximate value of the Hessian matrix discriminant of each pixel is:

[0126] det(H)=L xx L yy -(ωL xy ) 2

[0127] The filtering of the image by the box filter is transformed into the addition and subtraction operation of the sum of pixels between different areas on the image, which can be completed by just integrating the image.

[0128] ω represents the weighting coefficient, and det(H) represents the box filter response value in the area near the point.

[0129] The specific method of extracting candidate feature points in the scale control using the SURF algorithm is:

[0130] In order to obtain sampling points of different scales, it is necessary to construct the scale space of the image and extract feature points in the scale space. The scale space of SURF is composed of O groups and L layers. The same size of filters is used in different layers of the same group, but the blur coefficient of the filter gradually increases. According to the pyramid structure of the SURF algorithm, it keeps the original image unchanged and only changes the filter size;

[0131] The positioning of SURF feature points is to use neighborhood non-maximum suppression on the response images of feature points of different scales. In the constructed SURF pyramid, each pixel point passing through the Hessian matrix is ​​compared with 26 points in the two-dimensional image space and scale space neighborhood to select candidate feature points; then the three-dimensional linear interpolation method is used to locate the candidate points to obtain sub-pixel level feature points, thereby completing the extraction of feature points.

[0132] The main direction of the feature points in different material images is determined; in order to meet the rotation invariance, the main direction of the feature points must be determined. The specific method is: count the Haar wavelet features in the field of the feature points, in the field of the feature points, count the sum of the horizontal Haar wavelet features and the vertical Haar wavelet features of all points in a 60-degree sector, and get a response value for one sector; add up the response values ​​to form a vector, and select the longest vector direction as the main direction of the final feature point.

[0133] The specific method of constructing the corresponding SURF feature descriptor according to the feature points is:

[0134] Take a 4×4 rectangular area block around the feature point, and the direction of the obtained rectangular area is along the main direction of the feature point; divide the rectangular area block into 16 equal sub-areas, and count the Haar wavelet features of 25 pixels in the horizontal and vertical directions in each sub-area; the horizontal and vertical directions mentioned above are relative to the main direction;

[0135] The Haar wavelet features are: the sum of horizontal values ​​∑dx, the sum of vertical values ​​∑dy, the sum of absolute values ​​in the horizontal direction |∑dx|, and the sum of absolute values ​​in the vertical direction |∑dy|; the Haar wavelet value is used as the feature vector of each sub-block area, and a total of 64-dimensional vectors are used as the descriptors of the SURF features.

[0136] After the image of the material is collected, the image contour is extracted and the length, width and height of the material are calculated. The specific method is: in the learning stage, image 2 and background image 1 are used for difference, and in the counting stage, image 4 and background image 1 are used for difference to obtain a binary image, and then the binary image is subjected to a morphological opening operation to remove interference from component pins, and the opening operation adopts a cross convolution kernel;

[0137] The size of the cross convolution kernel is determined by the following method steps: S7-1, the circumscribed rectangle of the material is subtracted from the inscribed rectangle of the material to obtain the interference image of the material pins; S7-2, the minimum convolution kernel size d is used to erode the interference image by morphological corrosion operation; S7-3, the actual convolution kernel size is determined to be Len = 1.5×d; then the image contour is extracted, and the length l, width w, height h and area a of the material are counted.

[0138] The physical dimensions of the material, such as length, width, height and area, are matched with the information in the database file, and then the number of materials is counted according to the area method; the physical dimension matching conditions of the material are as follows:

[0139] S8-1. Horizontal placement judgment conditions:

[0140] L×0.8≤l≤L×1.2∧W×0.8≤w≤W×1.2∧A 平放 ×0.8≤a≤A 平放 ×1.2

[0141] S8-2, side placement judgment conditions:

[0142] L×0.8≤l≤L×1.2∧H×0.8≤h≤H×1.2∧A 侧放 ×0.8≤a≤A 侧放 ×1.2

[0143] S8-3, vertical placement judgment conditions:

[0144] W×0.8≤w≤W×1.2∧H×0.8≤h≤H×1.2∧A 竖放 ×0.8≤a≤A 竖放 ×1.2

[0145] Among them, L, W, H, A 平放 , A 侧放 , A 竖放 They respectively represent the length, width, height, horizontal area, side area and vertical area information of the file created in the learning phase; l, w, h and a respectively represent the length, width, height and area information of the material;

[0146] For materials that do not meet the above conditions and have two or more adhesion superpositions, distance transformation and watershed segmentation methods are used; distance transformation is that the gray value of each pixel in the binary image is the distance between the pixel and the nearest background pixel; the measurement metric used in this method is Euclidean distance; the main process of distance transformation is as follows:

[0147] Suppose a binary image contains a connected region S, which contains the target set O and the background set B. The distance transform is defined as follows:

[0148] D(p)=Min(disf(p,q)) p∈O,q∈B

[0149] Where D(p) represents the distance transformation value of point p; p represents a point in the target set O; q represents a point in the background set B;

[0150] S8-4, classifying target pixels in the image into internal points, external points and isolated points;

[0151] Taking the four-domain area of ​​the central pixel as an example, if the central pixel is the target pixel (value is 1) and the four neighbors are all target pixels (value is 1), then the point is an internal point. If the central pixel is the target pixel and the four neighbors are background pixels (value is 0), then the central point is an isolated point, as shown in the figure below. The target area points other than the internal points and isolated points are boundary points;

[0152] S8-5, calculate all the internal points and non-interior points in the image, the point sets are S1 and S2 respectively;

[0153] S8-6. For each internal point (x, y) in S1, use the distance formula disf() to calculate the minimum distance in S2. All minimum distances constitute a set S3.

[0154] S8-7, calculate the maximum value Max and the minimum value Min in S3;

[0155] S8-8. For each internal point, the calculation formula of the converted grayscale value G(x,y) is as follows:

[0156] G(x,y)=255×|S3(x,y)-Min| / |Max-Min|

[0157] Where S3(x,y) represents the shortest distance between the point (x,y) in S1 and S2;

[0158] S8-9, remain unchanged for isolated points;

[0159] In the distance transformation, the distance function disf() is selected as the Euclidean distance:

[0160]

[0161] Where x and y represent the coordinates of a point in the plane, p(x1,y1) and q(x2,y2) represent the coordinates of p and q respectively;

[0162] For materials that are stuck together and cannot be separated, first use the area method to count the number of them. The formula is as follows:

[0163]

[0164] Where n represents the number of statistics, a represents the total area of ​​the materials superimposed together, and A represents the area of ​​one material study;

[0165] When n=1 and l>L×1.2∨w>W×1.2, then n=2.

[0166] The specific steps of using the grey model to judge the stability of the results obtained based on the physical size matching method are as follows:

[0167] S9-1, Accumulation Generation

[0168] x (0) ={x (0) (1),x (0) (2),x (0) (3),…,x (0) (N)}

[0169] x (0) The result obtained from the size matching from step 4 to step 5;

[0170] x (0) (1) is the result of the first match from step 4 to step 5;

[0171] x (0) (2) is the result obtained from the second matching from step 4 to step 5;

[0172] x (0) (3) is the result obtained from the third matching from step 4 to step 5;

[0173] x (0) (N) is the result obtained from the Nth matching from step 4 to step 5;

[0174] Let x 1 (1) = x (0) (1)

[0175] x 1 (2) = x (0) (1)+x (0) (2)

[0176] x 1 (3) = x (0) (1)+x (0) (2)+x (0) (30,

[0177] So we get a new data sequence:

[0178] x 1 ={x 1 (1),x 1 (2),x 1 (3),…,(N)}

[0179] S9-2. Establishing the GM(1,1) model

[0180] x (0) (N)+kz 1 (N) = b is the basic form of the GM (1,1) model (N = 2, 3, ..., n), where k is called the development coefficient, z 1(N) is called the whitening background value, b is called the gray action amount; introducing the matrix formula, we can get:

[0181]

[0182] So, the GM(1,1) model x (0) (N)+kz 1 (N) = b can be expressed as Y = Bu, using the normal equation (B T B) -1 B T Y uses the principle of least squares to find the values ​​of k and b;

[0183] S9-3. Prediction

[0184] The corresponding whitening model is From this we can get

[0185]

[0186] Let Q = N + 1 to get the predicted value

[0187] Where N = 0, 1, 2, 3, ..., n-1; Q = N+1; Represents the prediction of the n-time cumulative sum;

[0188] S9-4. Inspection

[0189] The predicted value Generate using the cumulative subtraction method

[0190] Where N = 0, 1, 2, 3, ..., n-1; Q = N+1;

[0191] Represents the predicted value when the number of matches is n;

[0192] Calculate residuals

[0193] Where N = 0, 1, 2, 3, ..., n-1; Q = N+1;

[0194] Calculate the mean of the residuals

[0195]

[0196] Calculate x (0) Mean of (N)

[0197]

[0198] Calculate the original sequence x (0)The variance of (N) is Y1 and the variance of the residual e(N) is Y2

[0199]

[0200] Calculate the posterior difference ratio

[0201]

[0202] Calculating the probability of small errors

[0203]

[0204] When C<0.35 and P>0.95, the grey model has a good accuracy level;

[0205] When C<0.45 and P>0.8, the accuracy level of the grey model is qualified;

[0206] When C<0.5 and P>0.7, the accuracy level of the grey model is marginal;

[0207] When C≥0.65 and P≤0.7, the accuracy level of the grey model is unqualified;

[0208] When the grey model accuracy level is good after inspection, it means that the result obtained by the physical size matching method is stable.

[0209] In this embodiment:

[0210] Assume that there are N materials, and the number of materials is obtained through G physical size matching. If the material counting result N appears the same number of times and reaches the threshold M (M≥G / 2), it is recorded as counting stability, and the material counting result corresponding to the same number of times appears is taken as the final result.

[0211] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0212] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for counting electronic components based on physical size matching, characterized in that: This counting method includes two links: learning link and counting link; The learning process includes the following steps: Step 1: illuminate a light-transmitting empty tray with a uniform light source, use a dual-camera device to capture images, stitch the images, and store the images as background image 1; Step 2: After placing a quantity of materials ξ on an empty tray that is transparent to light, a dual-camera device is used to capture images, stitch the images, and store the images as image 2. Image 2 minus background image 1 generates foreground image 3; Step 3: Model the three postures of the material in the foreground image 3 and extract the contours, calculate the length, width, height and area physical size information of the material, and store the information in a database file. The three postures include a horizontal posture, a side posture and a vertical posture; The counting process comprises the following steps: Step 4: Place a quantity of β new materials on the tray, β>ξ; after the materials are shaken apart, use a dual-camera device to capture images, stitch the images, store the images as image 4, and subtract background image 1 from image 4 to generate foreground image 5; extract the image contour based on foreground image 5, and calculate the length, width, height and area physical dimensions of the materials; Step 5: The physical dimensions of length, width, height and area of ​​the materials calculated in step 4 are matched with the information in the database file described in step 3. The number of individual materials is first counted, and then the distance transformation and watershed segmentation methods are used to segment multiple materials that are close together or stacked together, and then the numbers are counted. For materials that are close together or stacked together and cannot be divided, the number of materials is counted according to the area method; Step 6: Use the grey model to determine whether the result obtained based on the physical size matching method is stable; if the result obtained based on the physical size matching method is unstable, repeat step 4 to recalculate the new length, width, height and area physical dimensions of the material; Step 7: Repeat step 5 until the number of material counting results that appear the same reaches the threshold value M, which is recorded as a stable counting situation, and the material counting results corresponding to the number of times that appear the same are taken as the final results; In step 5, the length, width, height and area physical dimensions of the material described in step 4 are matched with the information in the database file described in step 3, and then the number of materials is counted according to the area method; the material physical dimension matching conditions are as follows: S8-1. Horizontal placement judgment conditions: L×0.8≤l≤L×1.2∧W×0.8≤w≤W×1.2∧A 平放 ×0.8≤a≤A 平放 ×1.2 S8-2, side placement judgment conditions: L×0.8≤l≤L×1.2∧H×0.8≤h≤H×1.2∧A 侧放 ×0.8≤a≤A 侧放 ×1.2 S8-3, vertical placement judgment conditions: W×0.8≤w≤W×1.2∧H×0.8≤h≤H×1.2∧A 竖放 ×0.8≤a≤A 竖放 ×1.2 Among them, L, W, H, A 平放 , A 侧放 , A 竖放 They respectively represent the length, width, height, horizontal area, side area and vertical area information of the file created in the learning phase; l, w, h and a respectively represent the length, width, height and area information of the material; For materials that do not meet the above conditions and have two or more overlapping parts, distance transformation and watershed segmentation methods are used; distance transformation is that the gray value of each pixel in the binary image is the distance between the pixel and the nearest background pixel.

2. The electronic component counting method based on physical size matching according to claim 1, characterized in that: The stitched image in step 1 uses the SURF algorithm to extract and match features of the images collected by the dual-camera device to generate an image with a higher resolution; The specific steps are as follows: S2-1, using SURF algorithm to extract feature points in different material images; S2-2, using the SURF algorithm to extract candidate feature points in the scale control, and then using the three-dimensional linear interpolation method to locate the candidate points; S2-3, determining the main direction of feature points in different material images; S2-4. Construct the corresponding SURF feature descriptor based on the feature points.

3. The electronic component counting method based on physical size matching according to claim 2, characterized in that: In S2-1, the specific method of extracting feature points in different material images using the SURF algorithm is: The SURF algorithm uses the Hessian matrix determinant to detect feature points. First, the image is convolved with a Gaussian template, and then a Hessian matrix with scale information is obtained at the pixel point. The expression is: Where L(x,y,σ)=G(x,y,σ)*I(x,y), G(x,y,σ) is a scale-variable Gaussian function, I(x,y) is the image, L xx (x,y,σ),L xy (x,y,σ),L yy (x, y, σ) are the second-order differentials in the x, xy, and y directions at the pixel point (x, y), respectively; therefore, the Hessian matrix determinant is ultimately simplified to: det(H)=L xx 50 yy -L xy 50 xy The approximate discriminant of the Hessian matrix for each pixel is: it(H)=L xx L yy -(ωL xy ) 2 ω represents the weighting coefficient, and det(H) represents the box filter response value in the area near the point.

4. The electronic component counting method based on physical size matching according to claim 3 is characterized in that: In S2-2, the specific method of extracting candidate feature points in the scale control using the SURF algorithm is: The positioning of SURF feature points is to use neighborhood non-maximum suppression on the response images of feature points of different scales. In the constructed SURF pyramid, each pixel point of the Hessian matrix is ​​compared with 26 points in the two-dimensional image space and scale space neighborhood to select candidate feature points; then the three-dimensional linear interpolation method is used to locate the candidate feature points to obtain sub-pixel level feature points.

5. The electronic component counting method based on physical size matching according to claim 2, characterized in that: In S2-3, the specific method for determining the main direction of the feature points in different material images is: Count the Haar wavelet features within the feature point area. Within the feature point area, count the horizontal Haar wavelet features and the sum of vertical Haar wavelet features of all points within a 60-degree sector, and get a response value for one sector; add up the response values ​​to form a vector, and select the longest vector direction as the main direction of the final feature point.

6. The electronic component counting method based on physical size matching according to claim 5, characterized in that: In S2-4, the specific method of constructing the corresponding SURF feature descriptor according to the feature points is: Take a 4×4 rectangular area block around the feature point, and the direction of the obtained rectangular area is along the main direction of the feature point; divide the rectangular area block into 16 equal sub-areas, and count the Haar wavelet features of 25 pixels in the horizontal and vertical directions in each sub-area; The Haar wavelet features are: the sum of horizontal values ​​∑dx, the sum of vertical values ​​∑dy, the sum of absolute values ​​in the horizontal direction |∑dx|, and the sum of absolute values ​​in the vertical direction |∑dy|; the Haar wavelet value is used as the feature vector of each sub-block area, and a total of 64-dimensional vectors are used as the descriptors of the SURF features.

7. The electronic component counting method based on physical size matching according to claim 1, characterized in that: In the steps 2 to 4, after collecting images of the material, the image contour is extracted and the length, width and height of the material are calculated. The specific method is: in the learning stage, image 2 and background image 1 are used for difference, and in the counting stage, image 4 and background image 1 are used for difference to obtain a binary image, and then the binary image is subjected to a morphological opening operation to remove interference from component pins, and the opening operation adopts a cross convolution kernel; The size of the cross convolution kernel is determined by the following method steps: S7-1, the circumscribed rectangle of the material is subtracted from the inscribed rectangle of the material to obtain the interference image of the material pins; S7-2, the minimum convolution kernel size d is used to erode the interference image by morphological erosion operation; S7-3. The actual convolution kernel size is determined to be Len=1.5×d; then the image contour is extracted, and the length l, width w, height h and area a of the material are counted.

8. The electronic component counting method based on physical size matching according to claim 7, characterized in that: The measurement metric used for distance transformation is Euclidean distance; the main process of distance transformation is as follows: Suppose a binary image contains a connected region S, which contains a target set O and a background set B. The distance transform is defined as follows: D(p)=Min(disf(p,q)) p∈O,q∈B Where D(p) represents the distance transformation value of point p; p represents a point in the target set O; q represents a point in the background set B; S8-4, classifying target pixels in the image into internal points, external points and isolated points; S8-5, calculate all the internal points and non-interior points in the image, the point sets are S1 and S2 respectively; S8-6. For each internal point (x, y) in S1, use the distance formula disf() to calculate the minimum distance in S2. All minimum distances constitute a set S3. S8-7, calculate the maximum value Max and the minimum value Min in S3; S8-8. For each internal point, the calculation formula of the converted grayscale value G(x,y) is as follows: G(x,y)=255×|S3(x,y)-Min| / |Max-Min| Where S3(x,y) represents the shortest distance between the point (x,y) in S1 and S2; S8-9, remain unchanged for isolated points; In the distance transformation, the distance function disf() is selected as the Euclidean distance: Where x and y represent the coordinates of a point in the plane, p(x1,y1) and q(x2,y2) represent the coordinates of p and q respectively; For materials that are stuck together and cannot be separated, first use the area method to count the number of them. The formula is as follows: Where n represents the number of statistics, a represents the total area of ​​the materials superimposed together, and A represents the area of ​​one material study; When n=1 and l>L×1.2∨w>W×1.2, then n=2.

9. The electronic component counting method based on physical size matching according to claim 1, characterized in that: In step 6, the grey model is used to determine whether the result obtained based on the physical size matching method is stable. The specific method is as follows: S9-1, Accumulation Generation x (0) ={x (0) (1),x (0) (2),x (0) (3),…,x (0) (N)} x (0) The result obtained from the size matching from step 4 to step 5; x (0) (1) is the result of the first match from step 4 to step 5; x (0) (2) is the result obtained from the second matching from step 4 to step 5; x (0) (3) is the result obtained from the third matching from step 4 to step 5; x (0) (N) is the result obtained from the Nth matching from step 4 to step 5; Let x 1 (1) = x (0) (1) x 1 (2)=x (0) (1)+x (0) (2), x 1 (3)=x (0) (1)+x (0) (2)+x (0) (3), So we get a new data sequence: x 1 ={x 1 (1),x 1 (2),x 1 (3),…,x 1 (N)} S9-2. Establishing the GM(1,1) model x (0) (N)+kz 1 (N) = b is the basic form of the GM (1,1) model (N = 2, 3, ..., n), where k is called the development coefficient, z 1 (N) is called the whitening background value, b is called the gray action amount; introducing the matrix formula, we can get: So, the GM(1,1) model x (0) (N)+kz 1 (N) = b can be expressed as Y = Bu, using the normal equation (B T B) -1 B T Y uses the principle of least squares to find the values ​​of k and b; S9-3. Prediction The corresponding whitening model is From this we can get Let Q = N + 1 to get the predicted value Where N = 0, 1, 2, 3, ..., n-1; Q = N+1; Represents the prediction of the n-time cumulative sum; S9-4. Inspection The predicted value Generate using the cumulative method Where N = 0, 1, 2, 3, ..., n-1; Q = N+1; Represents the predicted value when the number of matches is n; Calculate residuals Where N = 0, 1, 2, 3, ..., n-1; Q = N+1; Calculate the mean of the residuals Calculate x (0) Mean of (N) Calculate the original sequence x (0) The variance of (N) is Y1 and the variance of the residual e(N) is Y2 Calculate the posterior difference ratio Calculating the probability of small errors When C<0.35 and P>0.95, the grey model has a good accuracy level; When C<0.45 and P>0.8, the accuracy level of the grey model is qualified; When C<0.5 and P>0.7, the accuracy level of the grey model is marginal; When C≥0.65 and P≤0.7, the accuracy level of the grey model is unqualified; When the grey model accuracy level is good after inspection, it means that the result obtained by the physical size matching method is stable.

Images