Multi-hole visual dimension detection method based on feature clustering and kalman correction

Abstract

The application discloses a multi-round-hole visual size detection method based on feature clustering and Kalman correction, collects the image of the end face of the multi-round-hole target workpiece on a conveying belt, and carries out gray processing; performs ellipse detection based on a deep learning method, and divides out an interest region containing an ellipse feature in the image; extracts the contour of each interest region and carries out screening; adopts a DSRSAC method to perform ellipse parameter fitting, solves the geometric characteristic parameters of the ellipse, and quantifies the uncertainty of each geometric characteristic parameter; through uncertainty weighted clustering and feature screening, a geometric characteristic parameter set with the maximum similarity is obtained, an ellipse fitting result is output, the round-hole diameter is calculated, and optimization is carried out based on the same; all round-hole diameter detection results of the multi-round-hole end face of the target workpiece are output, and the size detection of the round hole is completed. The application is suitable for an industrial assembly line scene, continuously corrects the detected round-hole diameter, and improves the accuracy and reliability of the detection result.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and more specifically to a method for visual size detection of multiple circular holes based on feature clustering and Kalman correction. Background Technology

[0002] Machine vision-based intelligent inspection, utilizing image perception, feature extraction, and information analysis, automatically measures and judges the dimensions and geometric information of workpieces on assembly lines, playing a crucial role in intelligent manufacturing. However, in many mechanical manufacturing fields, unlike single-feature, small, and thin parts such as PCBs, for three-dimensional parts such as engines, gearboxes, and machine tools with multiple circular holes to be inspected on one side, the optical axis of the imaging component cannot simultaneously and perfectly coincide with the relative positions of the axes of all the holes. The small hole spacing makes it difficult to place cameras for each hole, making it difficult to simultaneously obtain ideal images of the contours of each hole. Due to the axis-shifting effect and perspective distortion, some or all of the holes exhibit elliptical features, posing a challenge to the realization of visual inspection.

[0003] Traditional edge detection methods use edge detection operators like Sobel and Canny to identify the edge contours of circular holes and obtain their geometric parameters. However, when dealing with elliptical features, these methods can only extract edge pixels and cannot directly fit a circle. Methods based on the Hough transform map edge points in image space to a parameter space and perform cumulative voting to find the most likely circle parameters. Since an ellipse has five degrees of freedom and the total number of pixels belonging to the ellipse is unknown, the Hough transform requires enormous computational resources and memory for fitting and filtering, making it difficult to meet real-time and accuracy requirements. Template matching methods compare the similarity between a known circular hole template image and the image to be detected to find the best matching position. However, this method struggles to handle changes in shape and size caused by changes in viewpoint, leading to matching failures or poor accuracy.

[0004] In recent years, deep learning-based methods have offered new insights into solving this problem. U-Net-based methods can accurately segment elliptical regions, providing a data foundation for ellipse fitting and size detection. Faster R-CNN and Mask R-CNN, based on elliptical region segmentation, directly obtain ellipse parameters through regression analysis. DetR and other Transformer-based paradigms treat elliptical target detection as a geometric prediction problem, extracting features through a CNN backbone and interactively outputting prediction results using the Transformer encoder features. Because the Transformer has a global attention mechanism, it can better understand the contextual relationships of all targets in the image, and its structure and post-processing are simpler than R-CNN methods. Overall, deep learning methods can segment and detect elliptical targets with high accuracy, and their robustness and accuracy often far surpass traditional methods in complex industrial scenarios with cluttered backgrounds, multiple targets, and high noise. However, the accuracy of these "end-to-end" methods depends on a large number of "pixel-level" labeled datasets in specific scenarios. Most networks are slow to train, have poor convergence performance, and low inference efficiency, resulting in high data and computation costs. Moreover, the pixel accuracy is still not ideal, making it difficult to meet the visual inspection needs of multi-hole workpiece end faces in industries with high flexibility, small datasets, complex lighting backgrounds, and fast production cycles. Summary of the Invention

[0005] The purpose of this invention is to perform visual dimension detection on the end faces of multi-circular holes in housing parts such as engines, gearboxes, and machine tools in industrial assembly line scenarios. This invention utilizes a single camera to vertically acquire images and proposes a visual dimension detection method for multi-circular holes based on feature clustering and Kalman correction. This method overcomes the shortcomings of existing traditional methods and deep learning methods, such as poor anti-interference ability, large amount of labeled data requirements, low detection efficiency, and insufficient accuracy leading to unsatisfactory robustness, accuracy, and detection efficiency.

[0006] To achieve the above functions, this invention designs a multi-circular hole visual size detection method based on feature clustering and Kalman correction. For the multi-circular hole end face of the target workpiece on the conveyor belt, the following steps S1-S9 are performed to complete the size detection of the circular holes:

[0007] Step S1: Using a camera with its optical axis perpendicular to the surface of the conveyor belt, images of the multi-hole end faces of the target workpiece at each moment are acquired and formed into a sequence of images arranged in chronological order.

[0008] Step S2: Convert each acquired image to grayscale;

[0009] Step S3: For each image, perform ellipse detection based on deep learning methods to segment multiple regions of interest containing elliptical features in the image;

[0010] Step S4: Extract the contours of each segmented region of interest, connect the contours of each region of interest to obtain a contour point chain, and filter the contours on the contour point chain.

[0011] Step S5: For all points on each selected contour, construct a parameter space, extract sampling points in the parameter space, substitute them into the elliptic curve equation, construct an elliptic model, use the DSRSAC method to fit the elliptic parameters, solve the geometric feature parameters of the ellipse, and quantify the uncertainty of each geometric feature parameter.

[0012] Step S6: For the contours selected in step S4, based on the uncertainty of the geometric feature parameters obtained in step S5, obtain the set of geometric feature parameters with the greatest similarity through uncertainty weighted clustering and feature selection, return to execute step S5, and output the geometric feature parameters of the obtained ellipse as the final ellipse fitting result.

[0013] Step S7: Based on the ellipse fitting results obtained in step S6, and combined with the camera's inherent parameters and calibration parameters, calculate the diameter of the circular hole;

[0014] Step S8: For each image in the sequence image set, optimize the calculated value of the diameter of the circular hole in the region of interest based on Kalman filtering;

[0015] Step S9: After the diameters of all holes in the regions of interest have been calculated and optimized, output the detection results of the diameters of all holes on the multi-hole end face of the target workpiece to complete the size detection of the holes.

[0016] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0017] 1. This invention significantly improves the robustness and accuracy of circular hole size detection under perspective distortion, with an average accuracy improvement of 61.8% compared to single-frame circle detection methods. It overcomes the limitations of existing methods under perspective distortion, which suffer from inaccurate fitting under noise, texture, and lighting interference. By introducing uncertainty quantification and weighted parameter clustering mechanisms, it filters and weights contour chains based on the integrity of geometric information. This helps to effectively eliminate low-confidence noise and abnormal interference, focusing on segmented contour features with high quality and high similarity, thereby significantly improving the accuracy of ellipse parameter estimation.

[0018] 2. This invention addresses the jitter issue in dynamic pipeline detection results, improving the reliability of measurement results. With 12 time-series images as input, the average detection accuracy of the converged results improved by 19.6%, with a maximum improvement of 39.1%. To address the numerical fluctuation problem that easily occurs in continuous pipeline detection, this invention utilizes Kalman filtering technology to continuously correct the detection data in the time-series images. By establishing a mapping model that integrates historical observations and current measurements, the diameter of the circular hole is smoothed and optimally estimated. This helps eliminate random errors in single-frame image detection, ensures the stability of the continuous measurement process, and improves measurement accuracy.

[0019] 3. Balancing detection efficiency with the accuracy of physical measurements, the computation time is significantly shorter than manual measurement, making it suitable for the production rhythm of conventional manufacturing enterprises. This invention combines the efficient positioning capabilities of deep learning with the precise measurement capabilities of geometric vision. First, UNet is used to achieve rapid segmentation and coarse localization of the region of interest. Then, through inverse solving of the world coordinate system, the image pixel coordinates are accurately converted into actual physical dimensions. This helps ensure the real-time requirements of assembly line operations, achieving high-precision dimensional measurement for industrial applications. Attached Figure Description

[0020] Figure 1 This is a flowchart of a method for visual size detection of multiple circular holes based on feature clustering and Kalman correction according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the detection scheme and detection principle provided by an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of image acquisition and region of interest segmentation provided according to an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of contour feature screening based on principal component analysis according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of ellipse parameter estimation and optimization based on uncertainty clustering according to an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of the time-series Kalman correction of the hole size detection results provided in an embodiment of the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0027] The multi-hole visual size detection method based on feature clustering and Kalman correction provided in this invention targets the multi-hole end faces of target workpieces on conveyor belts, referring to... Figure 1 Perform the following steps S1-S9 to complete the size inspection of the circular hole:

[0028] Step S1: Using a camera with its optical axis perpendicular to the surface of the conveyor belt, images of the multi-hole end faces of the target workpiece at each moment are acquired and formed into a sequence of images arranged in chronological order.

[0029] Reference Figure 2 An imaging assembly including a camera, lens, and light source is set up, with the camera's optical axis perpendicular to the conveyor belt surface. The camera's intrinsic and extrinsic parameters are calibrated. The target workpiece's travel path is completely contained within the camera's field of view. Once the workpiece enters the camera's field of view, images are acquired at regular time intervals until the workpiece leaves the camera's field of view. The actual length of the camera's field of view along the conveyor direction is calibrated as L, the height of the imaging plane from the conveyor belt plane is h0, and the height of the target workpiece's inspected end face from the reference plane is h. Once the target workpiece is fully within the camera's field of view, images are acquired in a time series... Image acquisition is performed on the device to obtain images containing... Image sequence of images .

[0030] Step S2: Convert each acquired image to grayscale;

[0031] If an RGB camera is used, the current image is converted to grayscale. Assume the current image index is k, and the grayscale image I... k =0.299R k +0.587G k +0.114B k , where R k G k B k These are the R, G, and B channels of the original image. Low-pass filtering and histogram equalization are performed on the acquired or grayscaled image to improve the signal-to-noise ratio and enhance edge features.

[0032] Step S3: For each image, perform ellipse detection based on deep learning methods to segment multiple regions of interest containing elliptical features in the image;

[0033] In step S3, a deep learning method is used to roughly label and segment the region containing the ellipse. In this embodiment, the deep learning method can be UNet, RCNN, or YOLO model. Figure 3Images of the target workpiece captured during its movement on the assembly line are shown. Regions of interest (ROIs) segmented using deep learning methods are represented by red rectangles. Clearly, these ROIs are not pixel-level precise segments of the holes to be detected; it is sufficient to ensure that the hole image is within the regions. Subsequent computational steps will be performed within these ROIs.

[0034] Step S4: Extract the contours of each segmented region of interest, connect the contours of each region of interest to obtain a contour point chain, and filter the contours on the contour point chain.

[0035] The specific steps of step S4 are as follows:

[0036] Step S4.1: For each segmented region of interest, use the Canny edge detection operator to extract the edge features of each region of interest and obtain the contour of each region of interest.

[0037] Step S4.2: Obtain the connected chain of contour points using the Moore-Neighbor contour tracking algorithm;

[0038] Step S4.3: Due to the influence of noise, reflection, illumination, and other interferences, the obtained multiple contour point chains have strong uncertainties in their ability to track the real contour and their dependence on it. Directly applying them to ellipse fitting or parameter estimation will not yield results or will introduce large errors. Therefore, it is necessary to use principal component analysis to extract the geometric dispersion characteristics of the contour point chains for screening and identification.

[0039] Among all contour point chains obtained in the region of interest, elliptical arc segments with smaller central angles and shorter arc lengths may have an infinite number of similar feasible solutions, greatly reducing the reliability of the detection results. For the l-th contour containing n points, the second-order scatter matrix is ​​constructed as follows:

[0040] ;

[0041] in, Calculate the square matrix for the coordinates of the i-th point on the contour. All eigenvalues; due to It is an n×6 matrix. It is a 6×6 square matrix with 6 eigenvalues, arranged in descending order as follows: ;use Assess the information completeness of the contour when When this condition is met, it indicates that the contour contains a lot of geometric information, and the contour is retained. When the condition is met, it indicates that the contour is an ill-conditioned contour and should be discarded. The preset threshold is set in the embodiment. .

[0042] Reference Figure 4Two representative regions of interest will be selected for detailed explanation. Edge features are obtained using the Canny edge detection operator, as shown below. Figure 4 As shown in (a) and (b) in the figure. Edge features with connectivity are obtained using the Moore-Neighbor contour tracking algorithm. Figure 4 (c) and (d) are represented by different colors. For example... Figure 4 In the regions of interest shown in (a) and (c), the surface texture around the circular hole to be detected is relatively smooth with low density. However, this circular hole is a threaded hole, and the internal threads significantly interfere with the accurate extraction of the contour. Figure 4 In the region of interest shown in (d), the interior of the circular hole to be detected only has a small amount of interference from the intersecting holes and the step plane. However, the outer region of the circular hole has high roughness and numerous textures. The contour of the chamfer and the shadows formed by the illumination on the chamfer surface significantly interfere with the accurate extraction of the circular hole contour. Therefore, it is difficult to obtain accurate results by directly using the extracted contour for ellipse fitting.

[0043] Using the principal component analysis method proposed in step S4 to... Figure 4 The contours in (c) and (d) are filtered. After constructing the second-order information scatter matrix, the second-order information completeness of each contour is obtained. The information completeness distribution of all contours in the two regions of interest is as follows: Figure 4 As shown in (e) and (f), the second-order information completeness varies greatly among different contour point chains, and contours with high information completeness are very limited. The extracted information completeness is higher than... The contour point chain, the result is as follows Figure 4 As shown in (g) and (h). Compare before and after step S4. Figure 4 (c) and (d) in the middle and after processing Figure 4 In the algorithm, contours (g) and (h) that are short in length, have small curvature, and have extremely complex geometric information that are difficult to represent with second-order expressions have been eliminated, which improves the reliability and accuracy of subsequent ellipse fitting.

[0044] Step S5: For all points on each selected contour, construct a parameter space, extract sampling points in the parameter space, substitute them into the elliptic curve equation, construct an elliptic model, use the DSRSAC method to fit the elliptic parameters, solve the geometric feature parameters of the ellipse, and quantify the uncertainty of each geometric feature parameter.

[0045] RANSAC is a commonly used parameter estimation method, but it often suffers from low computational efficiency in environments with unknown noise and high outlier rates. Furthermore, in ellipse fitting scenarios, random sampling can lead to overly concentrated point sets and decreased reliability. Meanwhile, existing sampling techniques such as farthest-point sampling and maximum / minimum distance sampling, while beneficial for improving point set coverage and fitting reliability, can affect the diversity of the sampled point set, resulting in slower convergence. Therefore, this invention proposes the DSRSAC method to address this issue. It improves the traditional RANSAC method by utilizing density-supplemented resampling, maintaining point set diversity while increasing sampling point set coverage and solution reliability, thus enhancing the algorithm's convergence efficiency.

[0046] The specific steps of step S5 are as follows:

[0047] Step S5.1: For the l-th contour containing n points, parameterize all points according to their indices, with the parameter corresponding to the i-th point being i / n; if it is a closed contour, randomly select a point as the starting point and parameterize the positions of all other points, and construct a parameter space based on all parameterized points;

[0048] Step S5.2: Use Gaussian random selection to randomly select a point in the parameter space as the initial point;

[0049] Step S5.3: Based on the existing n sampling points in the parameter space, perform Gaussian kernel density estimation at any position; if the existing sampling point set is {t1,t2,…,t…} n}, standard deviation of sample points It can be represented as:

[0050] ;

[0051] The density at any position t can be calculated using the following formula. :

[0052] ;

[0053] According to density The undersampling weight is calculated according to the following formula. :

[0054] ;

[0055] In the formula, To prevent small amounts from being divided by zero, in the examples ;

[0056] Step S5.4: Undersampling weights This helps prevent new sampling points from falling near existing sampling points. To ensure the overall coverage of the sampling point set, the undersampling weights are further weighted by distance. Based on the undersampling weight function... Constructing feature distances based on location scale t Solve for the weighted undersampling density To perform sampling density correction, as shown in the following formula:

[0057] ;

[0058] Step S5.5: In the parameter space, with undersampling density Given the sampling probability, extract the next sampling point; if the total number of sampling points is less than 5, return to step S5.3, otherwise proceed to step S5.6;

[0059] Step S5.6: Substitute the obtained 5 sampling points {(x1,y1),(x2,y2),…,(x5,y5)} into the elliptic curve equation:

[0060] ;

[0061] Where x and y are the coordinates of each sampling point, and A, B, C, D, E, and F are the ellipse parameters;

[0062] The linear equations in matrix representation are obtained:

[0063] ;

[0064] Where M is the coefficient matrix;

[0065] If using The left singular vector matrix representing the coefficient matrix M is denoted by . The right singular vector matrix representing the coefficient matrix M is used to perform SVD decomposition on the coefficient matrix M of the linear equation, as shown in the following equation:

[0066] ;

[0067] Get The last column corresponds to the direction of the minimum singular value and is used as the ellipse parameters [A, B, C, D, E, F]; if the ellipse parameters satisfy the ellipse condition... If the sample point set is selected, proceed to step S5.7; otherwise, discard the sample point set and return to step S5.2.

[0068] Step S5.7: For any point (x, y), calculate the Sampson distance. Construct a distance metric between a point and an ellipse model:

[0069] ;

[0070] Calculate the Sampson distance between points in the entire contour chain; when a certain point... When this point is considered an interior point; The pixel distance threshold related to noise is set to 1 to 3 in this example;

[0071] Calculate the proportion of interior points in the current ellipse model; if the current iteration number is 1, record the current ellipse model and denote the current proportion of interior points as . If the proportion of interior points in the current model is higher than... If smaller, then update. The value is updated and the ellipse model is updated; after iterating a preset number of times, the iteration number is set to 1000 in this example; the optimal ellipse model is output.

[0072] Step S5.8: To construct a scale-invariant residual equation and avoid singularities in the Jacobian matrix, the constant term F in the ellipse parameters is fixed to 1. For any point (x) in the contour chain... i ,y i The fitting residual at that point The solution can be obtained using the following formula:

[0073] ;

[0074] The set of residuals at all n points variance for:

[0075] ;

[0076] The Jacobian equation for the entire profile chain is:

[0077] ;

[0078] The covariance matrix is ​​calculated using the following formula:

[0079] ;

[0080] The Jacobian matrix is ​​an n×5 matrix, and C is a 5×5 covariance matrix; the eigenvalues ​​on the diagonal of the covariance matrix C represent the uncertainties of the ellipse parameters [A,B,C,D,E], denoted as [C]. A C B C C C D C E ];

[0081] Step S5.9: Compared to the coefficients of the ellipse equation, ellipse parameters such as the coordinates of the center point and the lengths of the axes more fully reflect the geometric characteristics. Based on the ellipse parameters, the geometric characteristic parameters of the ellipse are solved as follows:

[0082] ;

[0083] ;

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] ;

[0090] in, The major axis of the ellipse, Let be the minor axis of the ellipse. Let θ be the center coordinates of the ellipse, and θ be the rotation angle of the ellipse's major axis relative to the x-axis. (Intermediate parameters...) This represents the stretching amount after the ellipse has been rotated. This is a constant correction amount after translating the center point of the ellipse to the origin;

[0091] Step S5.10: Calculate the Jacobian matrix for covariance propagation. The partial derivatives of each geometric parameter in the expression with respect to each equation coefficient:

[0092] ;

[0093] The covariance matrix of the geometric feature parameters is obtained through the error propagation formula. :

[0094] ;

[0095] because Given a 5×6 matrix and C is a 6×6 square matrix, then... It is a 5×5 square matrix; if the covariance matrix is... Describe the elements on the diagonal of the l-th contour point chain within the region of interest. For each parameter The variance of the geometric characteristic parameter is then used as a measure of uncertainty.

[0096] Step S6: For the contours selected in step S4, based on the uncertainty of the geometric feature parameters obtained in step S5, obtain the set of geometric feature parameters with the greatest similarity through uncertainty weighted clustering and feature selection, return to execute step S5, and output the geometric feature parameters of the obtained ellipse as the final ellipse fitting result.

[0097] In step S5, the geometric parameters and uncertainties corresponding to the contour point chains have been obtained. The elliptical contour to be determined must consist of one or more contour point chains with similar geometric parameters. Clustering is used to obtain the parameter set with the greatest similarity. Considering the uncertainty quantification feature in clustering helps to reduce the impact of geometric parameters with large uncertainties on the clustering results, focusing on the optimal contour set with lower uncertainty and higher significance of elliptical geometric features.

[0098] The specific steps of step S6 are as follows:

[0099] Step S6.1: Among the m contours retained in step S4, the normalization uncertainty of the points representing the geometric feature parameters of the l-th contour. As shown in the following formula, where The standard deviation parameters, obtained from the diagonal of the covariance matrix of the contour point chain i, respectively, characterize... Uncertainty of parameters:

[0100] ;

[0101] Construct uncertainty correlation weights for contours l and j As shown in the following formula:

[0102] ;

[0103] Construct the uncertainty-weighted distance between contour l and contour j As shown in the following formula:

[0104] ;

[0105] in, The norm function is used; in this example, the 2-norm is used.

[0106] Constructing a candidate contour set :

[0107] ;

[0108] ;

[0109] in, For the weighted adjacency distance threshold, This is the threshold for the number of adjacencies; For the weighted adjacent contour set of contour l, the function Represents the number of elements in the set; in this embodiment, we choose... , The weighted distance based on this association weight effectively reduces the equivalent distance of high-uncertainty parameters, improves the algorithm's confidence in low-uncertainty parameters, and avoids the impact of the local clustering effect of a large number of uncertain contour elements on the reliability of the calculation results.

[0110] Step S6.2: Assumption The coordinates of the starting point of the middle contour l are The coordinates of the termination point are The starting angle of contour l With termination angle Solve using the following formula, the range of all values ​​is [missing information]. :

[0111] ;

[0112] ;

[0113] Candidate contour set The points in the middle are adjusted according to the normalized uncertainty. Arrange them in ascending order, and select the corner coverage intervals sequentially from front to back according to the greedy principle. Find the largest set of non-overlapping contour point chains;

[0114] Step S6.3: Treat the selected set of contour point chains as a whole, return to step S5, output the geometric feature parameters of the obtained ellipse as the final ellipse fitting result; at the same time, obtain the geometric mean of the standard deviation of the geometric feature parameters. As a general measure of uncertainty:

[0115] ;

[0116] Geometric feature screening based on the corner coverage area helps to eliminate the influence of interfering visual elements such as artifacts, chamfers, and scratches that cause concentric ellipses. When the coverage areas overlap, only the contours with lower uncertainty are considered, which helps to improve the reliability of the ellipse fitting results.

[0117] Reference Figure 5 ,right Figure 5 Each selected contour shown in (a) and (b) undergoes ellipse parameter estimation and uncertainty quantification, transforming the ellipse equation parameters into geometric parameters and transferring the uncertainty of the equation parameters to the uncertainty of the geometric parameters. Further uncertainty-weighted contour clustering is performed using the method described in step S6. When each contour is used to describe an ellipse, the corresponding uncertainty parameters are as follows: Figure 5 As shown in (c) and (d), the ordinates are all logarithmic with base 10. In region of interest 1, the contour uncertainty parameter at index 5 is extremely small, indicating the existence of a highly deterministic elliptical contour, corresponding to... Figure 5 The outermost green outline in (a) is shown. Within region of interest 2, the outline with index 5 has relatively low uncertainty, the outline with index 4 has relatively high uncertainty, and the remaining outlines with indices 1, 2, and 3 have similar levels of uncertainty. Further identification is performed using uncertainty-weighted clustering, and the clustering results are as follows: Figure 5 As shown in (e) and (f), since the independent variables here are the ellipse's center coordinates, major axis, minor axis, rotation angle, and a 6-dimensional parameter of uncertainty, they are represented by two 3D graphs. The calculation results show that the contours in region of interest 1 do not exhibit a clear clustering tendency, while in region of interest 2, the information of three contours is relatively similar. Figure 5 The same color is used in (f) in the diagram.

[0118] Using the method described in step S6, contour filtering is performed based on the above clustering results. The contour containing the highest-certainty cluster in region of interest 1 is taken as the optimal contour set, and ellipse fitting is performed again. The resulting optimal ellipse is shown below. Figure 5 As shown in (g) in the diagram. The set of three contours contained in the class with the highest certainty in region of interest 2 is taken as the optimal contour set. After ellipse fitting, the optimal ellipse obtained is shown in the diagram. Figure 5 As shown in (h) in the diagram.

[0119] Step S7: Based on the ellipse fitting results obtained in step S6, and combined with the camera's inherent parameters and calibration parameters, calculate the diameter of the circular hole;

[0120] The ellipse obtained in steps S1-S6 is actually a perspective transformation of a circular hole in the world coordinate system. Each chord passing through the center of the ellipse is a diameter line passing through the center of the circle in the world coordinate system. Among them, the major axis of the ellipse has the largest length and the smallest compression, so it has the highest measurement accuracy. Therefore, the two endpoints of the major axis of the ellipse are used as reference points for coordinate transformation to solve for the measured value of the diameter of the circular hole.

[0121] The specific steps of step S7 are as follows:

[0122] Step S7.1: Calculate the coordinates of the two endpoints in the image coordinate system using the geometric feature parameters of the ellipse. :

[0123] ;

[0124] ;

[0125] Transform one of the endpoints to the direction vector in the camera coordinate system. , direction vector Transform to the world coordinate system to obtain the direction vector from the origin to that endpoint. ;in, The intrinsic parameter matrix of the camera and the extrinsic parameter matrix of the camera are respectively... , t;

[0126] The world coordinates of the camera center are ; the parameter form of a ray passing through the origin and its endpoint in the world coordinate system. Expressed as:

[0127] ;

[0128] in, Indicates the scale factor;

[0129] Step S7.2: The height of the plane containing the finally fitted ellipse from the imaging plane. As shown in the following formula:

[0130] ;

[0131] in, and These represent the world coordinates of the camera center. and the direction vector from the origin to the endpoint The component in the z-direction in a three-dimensional coordinate system;

[0132] The scaling factor is solved using the above formula. Substitute the value into After the expression, solve for the endpoints. True coordinates in the world coordinate system As shown in the following formula:

[0133] ;

[0134] The other end True coordinates in the world coordinate system for:

[0135] ;

[0136] Step S7.2: Since the ellipse is formed by perspective distortion through a circular aperture, the two endpoints of the major axis are the two ends of the diameter in the world coordinate system. The measured value of the diameter of the circular aperture is calculated as follows:

[0137] ;

[0138] in, This indicates the measured value of the diameter of the circular hole.

[0139] Step S8: For each image in the sequence image set, optimize the calculated value of the diameter of the circular hole in the region of interest based on Kalman filtering;

[0140] The imaging component continuously acquires images of the workpieces on the production line. Measurement values ​​for each region of interest in each image are obtained through steps S1-S7. The closer the workpiece is to the optical axis, the smaller the influence of perspective transformation, the less the deformation, and the lower the uncertainty and accuracy of visual measurement. Therefore, images acquired at different times do not have the same influence weight, and direct averaging will lead to significant errors.

[0141] For time series Image sequences acquired In the j-th region of interest, by time Images captured on The size of the round hole detected in the middle is Using the initial value, perform cyclical correction through the following steps S8.1-S8.6:

[0142] Step S8.1: Initialize the iteration number i to 1. Initial posterior estimation at time 1 The initial posterior variance of the estimation error ;

[0143] Step S8.2: Perform iterations, making the iteration number i = i + 1. Prior estimation of time Posterior estimate of the previous time step Same, that is The prediction process for detecting state covariance can be described as follows: ,in Let (i+1) be the prior variance. Let be the posterior variance at time (i). The uncertainty of the ellipse geometric feature parameters obtained by fitting in step S6;

[0144] Step S8.3: Utilize the pixel accuracy level calibrated at the factory of the industrial camera. Calculate Kalman gain ; The value of determines the reliability of the new measurement; if the prediction uncertainty is high, the new measurement will be largely adopted.

[0145] ;

[0146] Step S8.4: Transfer the new observation at time (i+1) Incorporating the existing prior estimates, the (i+1)th posterior estimate is calculated as follows:

[0147] ;

[0148] Step S8.5: Update the posterior variance at time (i+1) as follows: As new observational information is continuously incorporated into the observations, the uncertainty of the observations gradually decreases:

[0149] ;

[0150] Step S8.6: If there are still unused time-series images, return to step S8.2; if the workpiece has left the field of view and all time-series detection images have been used, output the current... The result of the detection of the diameter of the circular hole in the current region of interest.

[0151] Reference Figure 6 The two major axis endpoints of the ellipse were extracted, and the coordinates were transformed to the world coordinate system using the calibrated camera intrinsic and extrinsic parameter matrices. The distance between the two transformed points was used as the detection result of the circular hole size in the current region of interest at the current moment. The detection results for region of interest 1 and region of interest 2 were 14.05 mm and 9.83 mm, respectively. The detection results of the circular hole size in the two regions of interest were obtained by performing detection on all 12 images obtained after the workpiece entered the field of view and before it left the field of view. Figure 6 As shown in (a) and (b), the horizontal red dashed lines represent the nominal values ​​marked on the drawing, which are 8mm and 10mm respectively. Clearly, compared to the nominal values, the vision-based dimensional inspection method always carries a certain degree of error. Due to the significant perspective distortion when the workpiece is far from the optical axis, the image noise, interference, and clarity are all unsatisfactory, resulting in a generally large inspection error. Therefore, time 1 in the figure represents the position with the largest dimensional inspection error. As the workpiece gradually moves from the end far from the camera's optical axis to a position close to the optical axis, the error gradually decreases, and the measured value gradually stabilizes near the nominal value on the drawing. When the workpiece gradually moves away from the optical axis from the near end, the error begins to increase again, corresponding to the curve change trend after time 8 in the figure.

[0152] Processed using the temporal Kalman correction method described in step S8, the size correction result, Kalman gain, and variance changes under the input temporal image are as follows: Figure 6 As shown in (c) and (d) in the figure, it is clear that with the input of time-series images, the corrected size detection results gradually converge, the Kalman gain and measurement variance gradually stabilize, and the detection accuracy gradually improves.

[0153] On a workstation with a U9 285K processor, 64GB of memory, and a Win11 operating system, the average processing time after each input image frame is 0.65 seconds, far less than the manual measurement time based on vernier calipers or smooth limit gauges, which can meet the production cycle of a typical mechanical manufacturing enterprise. The final convergence values ​​of the circular hole dimensions in the two regions of interest are 8.02 mm and 10.05 mm, respectively, with relative errors of 0.25% and 0.5% compared to the nominal values ​​on the drawings. Manual measurement using a 50-division vernier caliper yielded results of 8.02 mm and 10.06 mm, respectively, with residuals of 0% and 0.10% for the method proposed in this invention. The visual inspection results are highly consistent with the nominal values ​​on the drawings and the manual measurement results, verifying the effectiveness and accuracy of the method proposed in this invention.

[0154] The change in the residual between the detected value and the current time step correction value is as follows: Figure 6 As shown in (e) and (f) in the diagram, in the two regions of interest, when the first detection result is input, the detected value is the same as the corrected value. In the subsequent input of 11 images, the detection accuracy improved by an average of 19.6% and a maximum of 39.1%. The final convergence value of the two regions of interest and the minimum relative error obtained from the detection of the single-frame image closest to the circle were reduced by 37.1% and 86.6%, respectively, verifying the superiority of the method of this invention compared to the single-frame circle detection image method.

[0155] Step S9: After the diameters of all holes in the regions of interest have been calculated and optimized, output the detection results of the diameters of all holes on the multi-hole end face of the target workpiece to complete the size detection of the holes.

[0156] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for visually detecting the size of multiple circular holes based on feature clustering and Kalman correction, characterized in that, For the multi-hole end face of the target workpiece on the conveyor belt, perform the following steps S1-S9 to complete the dimensional inspection of the holes: Step S1: Using a camera with its optical axis perpendicular to the surface of the conveyor belt, images of the multi-hole end faces of the target workpiece at each moment are acquired and formed into a sequence of images arranged in chronological order. Step S2: Convert each acquired image to grayscale; Step S3: For each image, perform ellipse detection based on deep learning methods to segment multiple regions of interest containing elliptical features in the image; Step S4: Extract the contours of each segmented region of interest, connect the contours of each region of interest to obtain a contour point chain, and filter the contours on the contour point chain. Step S5: For all points on each selected contour, construct a parameter space, extract sampling points in the parameter space, substitute them into the elliptic curve equation, construct an elliptic model, use the DSRSAC method to fit the elliptic parameters, solve the geometric feature parameters of the ellipse, and quantify the uncertainty of each geometric feature parameter. Step S6: For the contours selected in step S4, based on the uncertainty of the geometric feature parameters obtained in step S5, obtain the set of geometric feature parameters with the greatest similarity through uncertainty weighted clustering and feature selection, return to execute step S5, and output the geometric feature parameters of the obtained ellipse as the final ellipse fitting result. Step S7: Based on the ellipse fitting results obtained in step S6, and combined with the camera's inherent parameters and calibration parameters, calculate the diameter of the circular hole; Step S8: For each image in the sequence image set, optimize the calculated value of the diameter of the circular hole in the region of interest based on Kalman filtering; Step S9: After the diameters of all holes in the regions of interest have been calculated and optimized, output the detection results of the diameters of all holes on the multi-hole end face of the target workpiece to complete the size detection of the holes.

2. The method for visual size detection of multiple circular holes based on feature clustering and Kalman correction according to claim 1, characterized in that, The specific method for step S1 is as follows: The camera's optical axis is perpendicular to the conveyor belt surface, and the target workpiece's travel path is completely contained within the camera's field of view. The actual length of the camera's field of view along the conveyor direction is calibrated as L, the height of the imaging plane from the conveyor belt plane is h0, and the height of the target workpiece's inspected end face from the reference plane is h. Once the target workpiece is fully within the camera's field of view, in the time series... Image acquisition is performed on the device to obtain images containing... Image sequence of images .

3. The method for visual size detection of multiple circular holes based on feature clustering and Kalman correction according to claim 1, characterized in that, The specific steps of step S4 are as follows: Step S4.1: For each segmented region of interest, use the Canny edge detection operator to extract the edge features of each region of interest and obtain the contour of each region of interest. Step S4.2: Obtain the connected chain of contour points using the Moore-Neighbor contour tracking algorithm; Step S4.3: For the l-th contour containing n points, construct the second-order scatter matrix as follows: ； in, Calculate the square matrix for the coordinates of the i-th point on the contour. All eigenvalues; due to It is an n×6 matrix. It is a 6×6 square matrix with 6 eigenvalues, arranged in descending order as follows: ;use Assess the information completeness of the contour when When, preserve the outline, when When necessary, discard the outline; This is the preset threshold.

4. The method for visual size detection of multiple circular holes based on feature clustering and Kalman correction according to claim 1, characterized in that, The specific steps of step S5 are as follows: Step S5.1: For the l-th contour containing n points, parameterize all points according to their indices, with the parameter corresponding to the i-th point being i / n; if it is a closed contour, randomly select a point as the starting point and parameterize the positions of all other points, and construct a parameter space based on all parameterized points; Step S5.2: Use Gaussian random selection to randomly select a point in the parameter space as the initial point; Step S5.3: Based on the existing n sampling points in the parameter space, perform Gaussian kernel density estimation at any position; if the existing sampling point set is {t1,t2,…,t…} n }, standard deviation of sample points Represented as: ； The density at any position t can be calculated using the following formula. : ； According to density The undersampling weight is calculated according to the following formula. : ； In the formula, To prevent small amounts from being divided by zero; Step S5.4: Based on the undersampling weight function Constructing feature distances based on location scale t Solve for the weighted undersampling density To perform sampling density correction, as shown in the following formula: ； Step S5.5: In the parameter space, with undersampling density Given the sampling probability, extract the next sampling point; if the total number of sampling points is less than 5, return to step S5.3, otherwise proceed to step S5.6; Step S5.6: Substitute the obtained 5 sampling points {(x1,y1),(x2,y2),…,(x5,y5)} into the elliptic curve equation: ； Where x and y are the coordinates of each sampling point, and A, B, C, D, E, and F are the ellipse parameters; The linear equations in matrix representation are obtained: ； Where M is the coefficient matrix; If using The left singular vector matrix representing the coefficient matrix M is denoted by . The right singular vector matrix representing the coefficient matrix M is used to perform SVD decomposition on the coefficient matrix M of the linear equation, as shown in the following equation: ； Get The last column corresponds to the direction of the minimum singular value and is used as the ellipse parameters [A, B, C, D, E, F]; if the ellipse parameters satisfy the ellipse condition... If the sample point set is selected, proceed to step S5.7; otherwise, discard the sample point set and return to step S5.

2. Step S5.7: For any point (x, y), calculate the Sampson distance. Construct a distance metric between a point and an ellipse model: ； Calculate the Sampson distance between points in the entire contour chain; when a certain point... When this point is considered an interior point; The pixel distance threshold related to noise; Calculate the proportion of interior points in the current ellipse model; if the current iteration number is 1, record the current ellipse model and denote the current proportion of interior points as . If the proportion of interior points in the current model is higher than... If smaller, then update. The value is updated and the ellipse model is updated; after a preset number of iterations, the optimal ellipse model is output. Step S5.8: Fix the constant term F in the ellipse parameters to 1. For any point (x) in the contour chain... i ,y i The fitting residual at that point Solve using the following formula: ； The set of residuals at all n points variance for: ； The Jacobian equation for the entire profile chain is: ； The covariance matrix is ​​calculated using the following formula: ； The Jacobian matrix is ​​an n×5 matrix, and C is a 5×5 covariance matrix; the eigenvalues ​​on the diagonal of the covariance matrix C represent the uncertainties of the ellipse parameters [A,B,C,D,E], denoted as [C]. A C B C C C D C E ]; Step S5.9: Based on the ellipse parameters, solve for the geometric characteristic parameters of the ellipse as follows: ； ； ； ； ； ； ； ； in, The major axis of the ellipse, Let be the minor axis of the ellipse. Let θ be the center coordinates of the ellipse, and θ be the rotation angle of the ellipse's major axis relative to the x-axis. (Intermediate parameters...) This represents the stretching amount after the ellipse has been rotated. This is a constant correction amount after translating the center point of the ellipse to the origin; Step S5.10: Calculate the Jacobian matrix for covariance propagation. : ； The covariance matrix of the geometric feature parameters is obtained through the error propagation formula. : ； because Given a 5×6 matrix and C is a 6×6 square matrix, then... It is a 5×5 square matrix; if the covariance matrix is... Describe the elements on the diagonal of the l-th contour point chain within the region of interest. For each parameter The variance of the geometric characteristic parameter is then used as a measure of uncertainty.

5. The method for visual size detection of multiple circular holes based on feature clustering and Kalman correction according to claim 1, characterized in that, The specific steps of step S6 are as follows: Step S6.1: Among the m contours retained in step S4, the normalization uncertainty of the points representing the geometric feature parameters of the l-th contour. As shown in the following formula, where The standard deviation parameters, obtained from the diagonal of the covariance matrix of the contour point chain i, respectively, characterize... Uncertainty of parameters: ； Construct uncertainty correlation weights for contours l and j As shown in the following formula: ； Construct the uncertainty-weighted distance between contour l and contour j As shown in the following formula: ； in, Represents the norm function; Constructing a candidate contour set : ； ； in, For the weighted adjacency distance threshold, This is the threshold for the number of adjacencies; For the weighted adjacent contour set of contour l, the function Represents the number of elements in the set; Step S6.2: Assumption The coordinates of the starting point of the middle contour l are The coordinates of the termination point are The starting angle of contour l With termination angle Solve using the following formula, the range of all values ​​is [missing information]. : ； ； Candidate contour set The points in the middle are adjusted according to the normalized uncertainty. Arrange them in ascending order, and select the corner coverage intervals sequentially from front to back according to the greedy principle. Find the largest set of non-overlapping contour point chains; Step S6.3: Treat the selected set of contour point chains as a whole, return to step S5, output the geometric feature parameters of the obtained ellipse as the final ellipse fitting result; at the same time, obtain the geometric mean of the standard deviation of the geometric feature parameters. As a general measure of uncertainty: 。 6. The method for visual size detection of multiple circular holes based on feature clustering and Kalman correction according to claim 1, characterized in that, The specific steps of step S7 are as follows: Step S7.1: Calculate the coordinates of the two endpoints in the image coordinate system using the geometric feature parameters of the ellipse. : ； ； Transform one of the endpoints to the direction vector in the camera coordinate system. , direction vector Transform to the world coordinate system to obtain the direction vector from the origin to that endpoint. ;in, The intrinsic parameter matrix of the camera and the extrinsic parameter matrix of the camera are respectively... , t; The world coordinates of the camera center are ; the parameter form of a ray passing through the origin and its endpoint in the world coordinate system. Expressed as: ； in, Indicates the scale factor; Step S7.2: The height of the plane containing the finally fitted ellipse from the imaging plane. As shown in the following formula: ； in, and These represent the world coordinates of the camera center. and the direction vector from the origin to the endpoint The component in the z-direction in a three-dimensional coordinate system; The scaling factor is solved using the above formula. Substitute the value into After the expression, solve for the endpoints. True coordinates in the world coordinate system As shown in the following formula: ； The other end True coordinates in the world coordinate system for: ； Step S7.2: Calculate the measured value of the circular hole diameter using the following formula: ； in, This indicates the measured value of the diameter of the circular hole.

7. The method for visual size detection of multiple circular holes based on feature clustering and Kalman correction according to claim 1, characterized in that, In step S8, for the time series Image sequences acquired In the j-th region of interest, by time Images captured on The size of the round hole detected in the middle is Using the initial value, perform cyclical correction through the following steps S8.1-S8.6: Step S8.1: Initialize the iteration number i to 1. Initial posterior estimation at time 1 The initial posterior variance of the estimation error ; Step S8.2: Perform iterations, making the iteration number i = i + 1. Prior estimation of time Posterior estimate of the previous time step Same, that is The prediction process for detecting state covariance is described as follows: ,in Let (i+1) be the prior variance. Let be the posterior variance at time (i). The uncertainty of the ellipse geometric feature parameters obtained by fitting in step S6; Step S8.3: Utilize the pixel accuracy level calibrated at the factory of the industrial camera. Calculate Kalman gain : ； Step S8.4: Transfer the new observation at time (i+1) Incorporating the existing prior estimates, the (i+1)th posterior estimate is calculated as follows: ； Step S8.5: Update the posterior variance at time (i+1) as follows: ； Step S8.6: If there are still unused time series images, return to step S8.2; If the workpiece has left the field of view and all time-series detection images have been used, output the current... The result of the detection of the diameter of the circular hole in the current region of interest.

Images