A high-precision measurement method for three-dimensional dimensions of workpieces based on binocular vision

Abstract

The present invention belongs to the field of image measurement technology, and in particular, provides a high-precision three-dimensional workpiece measurement method based on binocular vision. The method comprises the following steps: Step 1: Binocular camera calibration: Based on the corner point information of the calibration plate itself in three-dimensional space and the information of the corresponding points in the two-dimensional image, a corresponding mathematical relationship is established to obtain the internal and external parameters of the camera system. The present invention uses a local threshold segmentation algorithm to better complete the final segmentation of the preprocessed workpiece image and compensate for the defects and burrs generated by the OTSU algorithm. To improve the robustness of the algorithm, polygon fitting is performed on the workpiece contour to minimize the impact of burrs. After background noise removal is performed on the binary workpiece image, the workpiece is completely separated from the background in both the left and right images, which is crucial for subsequent vertex detection.

Description

Technical Field

[0001] The present invention relates to the field of image measurement technology, and in particular to a high-precision measurement method for three-dimensional dimensions of a workpiece based on binocular vision. Background Art

[0002] Traditional measurement methods and equipment have shortcomings such as slow measurement speed, low efficiency, and deformation of the measurement force at the contact point. They can no longer meet the inspection requirements of modern production enterprises. There is an urgent need for a fast, efficient, and accurate three-dimensional dimension measurement technology for workpieces. Using two-dimensional images to obtain three-dimensional information of the object to be measured and then measuring it is undoubtedly a good way to solve this problem. Under the condition that the measurement environment remains unchanged, the method that can directly improve the accuracy is to improve the quality of the collected image. There are many factors that affect the quality of image acquisition, such as: the resolution of the image acquisition camera, the degree of lens distortion, and the choice of ambient light source. However, the cost of high-resolution cameras is too high. Therefore, this project chooses to improve the dimensional detection accuracy of machine vision measurement technology from the perspective of image processing to solve the problem of inaccurate measurement accuracy.

[0003] The Chinese patent publication number is "CN102628669A", and its name is "Dimension Measuring Device, Dimension Measuring Method and Program for Dimension Measuring Device". The method provides a dimension measuring device, a dimension measuring method and a program for the dimension measuring device. The dimension measuring device includes: an imaging part, which photographs a workpiece on a movable stage; a depth extension part, which performs depth extension on multiple workpiece images at different Z-direction positions in the movable stage to generate a depth-extended image; a main image display part, which displays the depth-extended image obtained by photographing the main workpiece on the screen as a main image; a measurement position information generation part, which specifies a position to be measured and a measurement method for the main image to generate measurement position information; an edge extraction part, which extracts an edge of the position to be measured from the depth-extended image obtained by photographing the workpiece based on the measurement position information; and a dimension value calculation part, which obtains the dimension value of the position to be measured based on the extracted edge. The measurement data obtained by this method has poor accuracy and does not meet the measurement accuracy requirements. At the same time, the implementation process is complex and inefficient. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In view of the deficiencies in the prior art, the present invention provides a high-precision measurement method for the three-dimensional dimensions of a workpiece based on binocular vision, which solves the problems raised in the above-mentioned background technology.

[0006] (2) Technical solution

[0007] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0008] A high-precision measurement method for three-dimensional dimensions of a workpiece based on binocular vision comprises the following steps:

[0009] Step 1: Binocular camera calibration: Based on the corner point information of the calibration plate in 3D space and the corresponding points in the 2D image, a corresponding mathematical relationship is established to obtain the internal and external parameters of the camera system.

[0010] Step 2, image preprocessing: The binocular camera captures the left and right images of the cylindrical workpiece, and the two images are enhanced by Laplace transform; then bilateral filtering is used for image filtering;

[0011] Step 3, foreground and background separation: perform image segmentation on the image preprocessed in step 2, and remove background noise from the segmented image;

[0012] Step 4, epipolar correction: perform epipolar correction on the segmented left and right images to make them on the same horizontal line;

[0013] Step 5, vertex detection: perform vertex detection on the left and right workpiece images output in step 4 to determine the coordinates of the vertices in the image;

[0014] Step 6, vertex matching and three-dimensional size calculation: perform vertex matching of the left and right images based on the feature values at the vertices of the workpiece, and then calculate the three-dimensional size of the workpiece using the least squares method using the vertex coordinates of the left and right images.

[0015] Furthermore, in the image preprocessing in step 2, the Laplace transform is used to enhance the illumination of the two images. The Laplace operator can change the contrast of the image, sharpen the edges, make the image clearer, and highlight the detailed texture of the image. The edge of the workpiece image obtained in this paper is the area where the grayscale jumps, so the Laplace operator template operation has a good effect on the subsequent foreground and background separation.

[0016] Furthermore, in step 3, the object target is separated from the background to reduce the interference of background pixels. The local threshold segmentation algorithm is selected for the workpiece image segmentation. Because the OTSU segmentation algorithm will produce defects and burrs, local threshold segmentation is selected for accurate image segmentation, and polygon fitting is performed on the workpiece contour to suppress edge burrs. The purpose of separating the background is achieved by comparing the original image with the edge contour filling image, removing background noise after segmentation, and improving measurement accuracy.

[0017] Furthermore, in step 5, the present invention improves the traditional Harris algorithm in vertex detection, such as its slow running speed and inaccurate corner point positioning. First, similar points are removed based on the binary image, and then the maximum suppression template is improved to accurately detect the corners and edges of the workpiece image, thereby improving the real-time performance of the algorithm.

[0018] Furthermore, in terms of vertex matching in step 6, an innovative logarithmic polar coordinate matching method based on the center of the workpiece contour is proposed to address the feature of the workpiece surface with few textures. This avoids the requirement of rich textures in traditional stereo matching algorithms and greatly reduces the algorithm complexity and mismatching phenomenon. This method provides a new idea for vertex matching of regular workpieces.

[0019] (3) Beneficial effects

[0020] Compared with the existing technology, the present invention provides a high-precision three-dimensional measurement method for workpieces based on binocular vision, which has the following beneficial effects:

[0021] The present invention uses a local threshold segmentation algorithm to better complete the final segmentation of the preprocessed workpiece image and compensates for the defects and burrs caused by the OTSU algorithm. To improve the robustness of the algorithm, polygon fitting is performed on the workpiece contour to minimize the impact of burrs. After background noise removal is performed on the binary workpiece image, the workpiece is completely separated from the background in both the left and right images, which is crucial for subsequent vertex detection.

[0022] In this invention, the traditional Harris algorithm is improved. First, similar points are removed based on the binary image, and then the maximum suppression template is improved to accurately detect the corners and edges of the workpiece image, thereby improving the real-time performance of the algorithm. The proposed improved algorithm can not only accurately detect corners, but also accurately locate edges, reduce the false detection rate and missed detection rate, and reduce the running time, which not only improves the accuracy but also enhances the real-time performance of the algorithm.

[0023] The logarithmic polar coordinate matching method based on the contour center of the present invention can stably and accurately match the vertices of the left and right images, avoiding the requirement of rich textures in traditional stereo matching algorithms and greatly reducing the algorithm complexity and mismatching phenomena. The algorithm is simple and efficient, has superiority, feasibility and strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of the present invention;

[0025] Figure 2 This is a flow chart of local threshold segmentation of the present invention;

[0026] Figure 3This is a flowchart of the steps of the improved Harris algorithm of the present invention;

[0027] Figure 4 This is a comparison chart of relevant indicators between the existing technology and the method proposed in this invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0029] Example 1

[0030] like Figure 1-4 As shown, an embodiment of the present invention proposes a high-precision measurement method for three-dimensional dimensions of a workpiece based on binocular vision, comprising the following steps:

[0031] Step 1: Binocular camera calibration: This uses the widely used Zhang camera calibration method. By detecting the corner points on a calibration plate, a mathematical relationship is established between the corner points of the calibration plate in 3D space and the corresponding points in the 2D image, thereby obtaining the intrinsic and extrinsic parameters of the camera system.

[0032] Step 2, image preprocessing: The binocular camera collects the left and right images of the cylindrical workpiece, and the two images are enhanced by Laplace transform; the Laplace template is moved on the original image, and when the value in the template coincides with a pixel, they are multiplied and summed, and the value is assigned to the pixel that coincides with the center of the template. For the pixels in the first row, last row and column of the image that cannot be operated above, the value is assigned to zero, and the Laplace template operation result is obtained. Because the Laplace operator is a second-order differential operator, the operator emphasizes the discontinuous grayscale part of the image pixel and slows down the continuous grayscale value part, which will produce a very obvious grayscale boundary, and then use Bilateral filtering is used for image filtering. Bilateral filtering is a nonlinear filtering algorithm developed based on Gaussian filtering and has improved its shortcomings. Bilateral filtering retains the advantage of Gaussian filtering using a weighted average method, that is, the weighted average of the grayscale values of neighboring pixels represents the grayscale value of the center pixel of the template. In addition, grayscale similarity information is also referenced. The basic principle of the algorithm is to multiply a spatial distance Gaussian function with a grayscale distance Gaussian function. The spatial distance refers to the Euclidean distance between the current point and the center point of the template, and the grayscale distance is the absolute value of the difference between the grayscale value of the current point and the grayscale value of the center of the template. Therefore, bilateral filtering can filter out noise while maintaining the image edge.

[0033] Step 3, foreground and background separation: perform image segmentation on the image preprocessed in step 2, and remove background noise from the segmented image. Local threshold segmentation is a method of segmentation based on the different grayscale characteristics of the image target using different thresholds. First, determine the optimal grayscale threshold and determine a neighborhood of a specific size. The neighborhood is centered on the current pixel. The segmentation threshold of the neighborhood is calculated based on the grayscale values of all pixels in the neighborhood. If the grayscale values of the target and background in the image are relatively close, the mean method can be used to calculate the local threshold. The value range of the threshold is the grayscale range of the image. Then, the threshold is compared with the size of each pixel in the image. Finally, the image pixels are classified according to the comparison result. The image is binarized by the local threshold segmentation method. The contour image of the workpiece image is obtained by finding the maximum contour in the binarized image. Polygon fitting is performed on the workpiece contour to suppress edge burrs. There is a special function approxPolyDP in OpenCV to implement polygon fitting. This function can achieve different fitting effects by setting the threshold. Usually, the threshold is selected as a specific percentage of the perimeter of the contour to be fitted. The purpose of separating the background is achieved by comparing the original image with the edge contour filling image.

[0034] Step 4, epipolar correction: epipolar correction is performed on the segmented left and right images so that the left and right images are on the same horizontal line. The commonly used epipolar correction algorithm is the Bouguet algorithm, and its algorithm idea is as follows: (1) Adjust the imaging planes of the left and right cameras of the binocular system. According to the rotation matrix obtained by the camera calibration in step 1, the rotation parameters of the two cameras in the binocular system are obtained respectively, and the imaging planes of the left and right cameras are adjusted to be parallel. (2) Adjust the optical axes of the left and right cameras of the binocular system. According to the translation matrix obtained by the camera calibration in step 1, the optical axes of the left and right cameras are adjusted to be parallel. (3) Perform image correction based on the obtained rotation parameters and translation parameters.

[0035] Step 5, vertex detection: The improved Harris algorithm is used to perform vertex detection on the left and right workpiece images output in step 4 to determine the coordinates of the vertices in the image. In order to reduce the amount of calculation of the corner response function R value, the present invention removes similar points on the image. First, the similar points are judged and removed; the gradient value of the image is calculated, and the difference operator is used for Gaussian smoothing to remove noise; the autocorrelation matrix M is calculated, and the elements a, b, and c of the autocorrelation matrix are calculated. The response function R value is calculated for the pixel points at the junction of the target and the background after the similar points are removed. If the function R is much larger than the threshold, it is a corner point; when the function R<0, that is, R is negative, it is detected as an edge, and non-maximum suppression is performed. When performing local non-maximum suppression, the traditional Harris operator uses a 3×3 suppression template. Although a single suppression template can accurately detect corner points, it will mistakenly eliminate some edge points, resulting in breakpoints in the detected edges. To address the above problems, the present invention proposes to use 1×3 and 3×1 templates to suppress edges in the x and y directions, and use a 3×3 suppression template for corner points, so as to obtain more accurate and continuous edges and candidate corner points. The obtained candidate corner points are traversed. If there is more than one corner point within the 7×7 neighborhood of a candidate corner point, the arithmetic average of these candidate corner points in the neighborhood is calculated to accurately locate the position of the final corner point, thereby obtaining the final true corner point.

[0036] Step 6, vertex matching and three-dimensional size calculation. The vertices of the left and right images are matched according to the feature values at the vertices of the workpiece. The minimum circumscribed rectangle surrounding the contour vertices is first obtained using the rotation algorithm, and then the logarithmic polar coordinate system is established with the geometric center of the minimum circumscribed rectangle as the reference origin. The polar coordinates between the vertices are scattered and the intervals are large, which can clearly match the vertices in the left and right images from the image. The vertex matching method in this paper adopts a brute force loop matching method, that is, the vertex to be matched in the left image is selected, the logarithmic polar coordinate values of the vertices in the right image are traversed, and the point with the closest logarithmic polar coordinate values of the vertices in the left and right images is found. The two vertices are considered to match, and then the vertex coordinates of the left and right images are obtained using the least squares method to calculate the three-dimensional size of the workpiece.

[0037] Example 2

[0038] In step 1, the Camera Calibration toolbox in Matlab 2018 is used to calibrate the monocular camera. The calibration steps are as follows: (1) the software reads pictures of chessboard paper in different poses and detects corner points; (2) after checking the corner points of each picture, click the Options button to choose to calculate the second-order or third-order radial distortion coefficient, and then click the Calibrate button to obtain the camera parameters of the camera. The left and right cameras are calibrated according to the above calibration process for multiple times, and the one with the smallest error is selected as the calibration result of the binocular system. The left and right images of the calibration plate with different poses collected by the binocular system at the same time are analyzed to obtain the calibration result of the binocular system. The calibration steps are similar to those of the single-target calibration, so they are not repeated here. Finally, the internal and external parameters of the camera are obtained through two camera calibrations.

[0039] In step 2, the binocular camera collects the left and right images of the cylindrical workpiece, and the two images are enhanced by Laplace transform. The illumination enhancement formula is:

[0040]

[0041] Among them, g(x,y) is the transformed image and f(x,y) is the original image; c is a coefficient, and its value is related to the template center value. When the center value is positive, c = -1, otherwise c = 1.

[0042] Then bilateral filtering is used to perform image filtering. Assume that the grayscale value of the center pixel (i, j) of the original image is f(i, j), and the grayscale value of its neighboring pixel (k, l) is f(k, l). The formula of the bilateral filtering algorithm is:

[0043]

[0044] In step 3, the local threshold segmentation method is used to perform a threshold segmentation experiment on the image. Assume that the threshold of the image is T, f(x, y) is the grayscale value at the point (x, y), and the height and width of the local neighborhood window are represented by H and W respectively. The calculation formula is:

[0045]

[0046] Among them, C is a constant, which is used to adjust the calculated threshold T to achieve the best local threshold segmentation effect. Thresholds of 30, 50 and 70 are selected for experiments. Local threshold segmentation requires repeated experiments to determine the optimal threshold. Threshold 70 is the optimal threshold with the best segmentation effect. The preprocessed workpiece image is segmented using the local threshold segmentation algorithm. It can be seen from the local enlarged image that the local threshold segmentation better completes the final segmentation and makes up for the defects and burrs caused by the OTSU algorithm. 0.1% of the contour circumference is selected as the threshold to fit the left and right workpiece contours. By color filling the left and right workpiece contour fitting results, a filled image containing only the workpiece can be obtained. By comparing the original image with the filled image, the workpiece can be separated from the original image.

[0047] After the left and right images are corrected using the Bouguet algorithm in step 4, the left and right images are substantially on the same horizontal line.

[0048] In the step 5, the left and right workpiece images output in step 4 are subjected to vertex detection using an improved Harris algorithm to determine the coordinates of the vertices in the image. Similar points are first judged and removed; the gradient value of the image is calculated, and Gaussian smoothing is performed using a differential operator to remove noise; the autocorrelation matrix M is calculated, and the elements a, b, and c of the autocorrelation matrix are calculated. The response function R value is calculated for the pixel points at the junction of the target and the background after the similar points are removed. If the function R is much larger than the threshold, it is a corner point; when the function R<0, that is, R is negative, the detected pixel is an edge, and non-maximum suppression is performed. When performing local non-maximum suppression, two templates of 1×3 and 3×1 are used to suppress the edges in the x and y directions, and a 3×3 suppression template is used for the corner points, so as to obtain more accurate and continuous edges and candidate corner points. The obtained candidate corner points are traversed. If there is more than one corner point in the 7×7 neighborhood of a candidate corner point, the arithmetic average of the candidate corner points in the neighborhood is calculated to accurately position the final corner point, thereby obtaining the final true corner point.

[0049] In step 6, the initial minimum bounding rectangle is determined, a pixel point in the contour is selected as a vertex, its maximum and minimum values in the horizontal and vertical directions are determined, and the rectangle formed by the connection is used as the initial reference value of the minimum bounding rectangle, the angle of boundary rotation is determined, and the image is rotated to obtain the contour vertex p. i (x i ,y i ) The angle θ between the line segment determined by i , then vertex p i (x i ,y i ) The coordinate after rotation is marked as p i '(x' i ,y'i ), then p i 'with p i The transformation formula is as follows:

[0050]

[0051] Determine the minimum enclosing rectangle boundary, record the maximum and minimum values of the vertices in the horizontal and vertical directions generated after each boundary rotation in the previous step, and calculate their areas. After rotating all boundary angles, select the enclosing rectangle with the smallest area as the final minimum enclosing rectangle. When the center point of the minimum enclosing rectangle of the outline is used as the origin of the logarithmic polar coordinate system, the polar coordinates of the vertices of the left and right images in the logarithmic polar coordinate system are scattered and the intervals are large. It is possible to clearly match the vertices in the left and right images from the image. Finally, the vertex coordinates of the left and right images are obtained using the least squares method to obtain the spatial coordinate q of the vertex.

[0052] Its formula is

[0053] q=(k T k) -1 k T l

[0054] in:

[0055]

[0056]

[0057]

[0058] The projection distance of the vertex on the Z axis is not uniform, so in order to reduce the error, the size calculation of the workpiece is performed by projecting the selected point on the X and Y planes to obtain the three-dimensional size of the workpiece.

[0059] This study used a set of 3D image processing algorithms to find the best way to measure the three-dimensional shape of an object and a set of image elements, and then compared the three-dimensional shape of the object using the 3D image processing algorithm. The results show that this method has a very good approximation performance, which is a very good method for measuring the three-dimensional shape of an object. But the proposed method cannot meet the requirements of the current method. Figure 4 As shown:

[0060] from Figure 4It can be seen from the results that the method proposed in the present invention has higher measurement speed and accuracy than the existing method. These indicators also further illustrate that the method proposed in the present invention has faster measurement speed and higher measurement accuracy.

[0061] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A high-precision three-dimensional measurement method for workpieces based on binocular vision, characterized by: The following steps are included: Step 1: Binocular camera calibration: Based on the corner point information of the calibration plate in 3D space and the corresponding points in the 2D image, a corresponding mathematical relationship is established to obtain the internal and external parameters of the camera system. Step 2, image preprocessing: The binocular camera captures the left and right images of the cylindrical workpiece, and the two images are enhanced by Laplace transform; then bilateral filtering is used for image filtering; Step 3, foreground and background separation: perform image segmentation on the image preprocessed in step 2, and remove background noise from the segmented image; Step 4, epipolar correction: perform epipolar correction on the segmented left and right images to make them on the same horizontal line; Step 5, vertex detection: perform vertex detection on the left and right workpiece images output in step 4 to determine the coordinates of the vertices in the image; In step 5, in order to address the shortcomings of the traditional Harris algorithm in vertex detection, such as slow running speed and inaccurate corner point positioning, the present invention improves the traditional Harris algorithm by first removing similar points based on the binary image and then improving the maximum suppression template to accurately detect the corners and edges of the workpiece image, thereby improving the real-time performance of the algorithm. Step 6, vertex matching and 3D size calculation: perform vertex matching of the left and right images based on the feature values at the workpiece vertices, and then calculate the 3D size of the workpiece using the least squares method using the vertex coordinates of the left and right images. In terms of vertex matching in step 6, an innovative logarithmic polar coordinate matching method based on the center of the workpiece contour is proposed to address the feature of the workpiece surface with little texture. This avoids the requirement of rich textures in traditional stereo matching algorithms and greatly reduces the algorithm complexity and mismatching. This method also provides a new idea for vertex matching of regular workpieces.

2. The method for high-precision measurement of three-dimensional dimensions of a workpiece based on binocular vision according to claim 1, characterized in that: In the image preprocessing in step 2, the Laplace transform is used to enhance the illumination of the two images. The Laplace operator can change the contrast of the image, sharpen the edges, make the image clearer, and highlight the detailed texture of the image. The edge of the workpiece image obtained in this paper is the area where the grayscale jumps, so the Laplace operator template operation has a good effect on the subsequent foreground and background separation.

3. The method for high-precision three-dimensional measurement of a workpiece based on binocular vision according to claim 1, characterized in that: In step 3, the object target is separated from the background to reduce the interference of background pixels. The local threshold segmentation algorithm is selected for the workpiece image segmentation. Because the OTSU segmentation algorithm will produce defects and burrs, local threshold segmentation is selected for accurate image segmentation. Polygon fitting is performed on the workpiece contour to suppress edge burrs. The purpose of separating the background is achieved by comparing the original image with the edge contour filling image, removing background noise after segmentation, and improving measurement accuracy.

Images