A multi-camera large size vision measurement method and system

Abstract

The application relates to a multi-camera large-size visual measurement method and system, which comprises the following steps: installing a plurality of cameras for large-size measurement; marking a group of three-coordinate marker points in the visual field of each camera on a calibration whiteboard by using a three-coordinate measuring machine, and recording the three-coordinate positions and sequence of each three-coordinate marker point; acquiring images of the three-coordinate marker points in the visual field of each camera on the calibration whiteboard by the camera, and obtaining the center positions of the regions where the three-coordinate marker points are located; obtaining the pose transformation matrix of all the cameras; adopting a sub-pixel edge detection algorithm based on Gaussian fitting on the images collected by the camera to obtain an edge point set, and fitting an edge straight line by using the edge point set; obtaining a straight line intersection point based on the edge straight line; converting the straight line intersection point to a world coordinate system based on the obtained pose transformation matrix; and obtaining a size based on the converted position. The application is used for high-precision large-size visual measurement.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of pole piece detection, and particularly relates to a multi-camera large-size visual measurement method and system. BACKGROUND

[0002] The pole piece is used for a lithium battery. The production and manufacturing of the lithium battery are connected by multiple process steps, including three major stages of pole piece manufacturing, battery assembly and liquid injection. Each stage can be divided into several key processes, and each step has a great influence on the performance of the battery. The process of pole piece manufacturing mainly includes coating, baking, rolling, laser cutting, high-speed cutting, laminating and packaging.

[0003] In order to ensure that the laser cutting can produce pole pieces meeting the process requirements, the width of the pole piece in the process needs to be measured to prevent the cutting knife from cutting off. However, the accuracy of the scheme for measuring the width of the pole piece in the related art is insufficient, resulting in problems in the performance of the lithium battery generated based on the corresponding pole piece width measurement.

[0004] In the traditional detection method, when the width of the pole piece is large, in order to improve the detection accuracy, multiple large field of view high precision cameras are needed to fly to detect the width and length of the pole piece, including scenes with overlapping areas of multiple cameras and scenes without overlapping areas of multiple cameras. For the scene with overlapping areas of multiple cameras, since there is a common area between multiple images, image stitching can be performed through feature point detection, so that the size of the object can be obtained. For the scene without overlapping areas of multiple cameras, the local area positions of both ends of the object to be measured can be obtained through multiple cameras, and the object can be quickly measured by combining the pose parameters of the double cameras (including camera intrinsic parameters and camera extrinsic parameters), which is very suitable for online rapid batch detection.

[0005] Among them, the multi-camera calibration technology is the key to affecting the detection accuracy. The existing multi-camera calibration method basically adopts a checkerboard calibration board for calibration. The calibration using the checkerboard calibration board has the following several concentrated disadvantages.

[0006] (1) conventional calibration board precision is limited, usually the calibration board precision is 0.01mm, for um level measurement, the calibration precision is limited; (2) the calibration board is usually in the form of chessboard or circle dot marking, the size of the chessboard or circle dot is usually mm level, for high precision measurement system, in order to reduce the influence of lens distortion, usually far focus lens is used, the camera field of view is small, it is difficult to find enough number of calibration points of the calibration board in the field of view, which affects the calibration precision; (3) for large size object detection, the precision and size of the calibration board are usually difficult to be considered, it is difficult to establish the world coordinate system through a single calibration board for the detection area covered by multiple cameras, and the transformation between the multi-camera and the world coordinate system cannot be realized; (4) the calibration board has a certain thickness, the high precision measurement system usually uses far focus lens to eliminate distortion, the depth of field of the far focus lens is small, and the thickness of the calibration board causes a certain deviation between the best measurement distance plane during calibration and the best measurement distance plane during actual measurement, which affects the final precision. SUMMARY

[0007] In order to solve the above technical problems, one of the present application is to provide a multi-camera large size visual measurement method, which combines a three coordinate measuring machine to calibrate multiple cameras, obtains the pose transformation matrix of high precision cameras, improves the calibration precision, and further improves the measurement precision of the size.

[0008] In order to solve the above technical problems, the present application proposes the following technical solutions:

[0009] A multi-camera large size visual measurement method, comprising the following steps:

[0010] S1: install multiple cameras for large size measurement, wherein the fields of view of the cameras have no overlapping area;

[0011] S2: use a three coordinate measuring machine to mark a group of three coordinate markers on a calibration whiteboard, the three coordinate markers are respectively in the fields of view of the cameras, and record the three coordinate positions and sequence of the three coordinate markers, the size of the calibration whiteboard can cover the fields of view of the installed cameras and is within the range of the three coordinate measuring machine;

[0012] S3: the camera acquires the image of the three coordinate markers in the field of view of the camera on the calibration whiteboard, obtains the area where the three coordinate markers are located, and obtains the center position of the area;

[0013] S4: use the three coordinate positions of the three coordinate markers in the field of view and the corresponding center positions of the areas to calculate the pose transformation matrix between the pixel coordinate system of the camera and the world coordinate system of the three coordinate measuring machine;

[0014] S5: repeat S3 and S4 until the pose transformation matrix of all cameras is obtained;

[0015] S6: adopt a sub-pixel edge detection algorithm based on Gaussian fitting to the image collected by the camera to obtain an edge point set, and fit an edge straight line using the edge point set;

[0016] S7: obtain a straight line intersection point based on the edge straight line;

[0017] S8: convert the straight line intersection point to a world coordinate system based on the pose transformation matrix obtained in S5;

[0018] S9: obtain a size based on the converted position.

[0019] In some embodiments of the present application, the three-coordinate measuring machine uses a standard micro-circular probe contact method with a marker color to mark three-coordinate marker points.

[0020] In some embodiments of the present application, the camera in S3 collects images of three-coordinate marker points in its field of view on the calibration whiteboard, and obtains regions of each three-coordinate marker point, specifically:

[0021] perform binaryzation processing on the image;

[0022] extract regions of each three-coordinate marker point;

[0023] perform circular fitting on the regions to obtain circular spot regions corresponding to each three-coordinate marker point.

[0024] In some embodiments of the present application, the three-coordinate marker points recorded in S2 include a first group of marker points used to calculate the pose transformation matrix and a second group of marker points used to verify the calculated pose transformation matrix.

[0025] In some embodiments of the present application, the lens of the camera is a telecentric lens.

[0026] In some embodiments of the present application, in S4, the three-coordinate positions of the three-coordinate marker points in the field of view and the center positions of the corresponding regions are used to solve the pose transformation matrix by SVD.

[0027] In some embodiments of the present application, when the large size is the large size of the pole piece, there is at least one vertex of the pole piece in the field of view of the camera.

[0028] The present application also relates to a multi-camera large-size visual measurement system, comprising:

[0029] a plurality of cameras installed for large-size measurement, wherein the fields of view of the cameras have no overlapping regions;

[0030] a calibration whiteboard with a size capable of covering the fields of view of the installed cameras;

[0031] A coordinate measuring machine, the size of the calibration whiteboard is within the range of the coordinate measuring machine, the coordinate measuring machine marks a set of three coordinate markers in each camera field of view on the calibration whiteboard, and records the three coordinate positions and sequence of each three coordinate marker;

[0032] A first acquisition unit, each camera respectively acquires images of three coordinate markers in the field of view of the camera on the calibration whiteboard, and for each camera, acquires the area of each three coordinate marker in the image and the area center position of each area;

[0033] A first calculation unit, for each camera, calculates the pose transformation matrix between the pixel coordinate system of the camera and the world coordinate system of the coordinate measuring machine using the three coordinate positions of the three coordinate markers in the field of view and the corresponding area center positions;

[0034] A straight line acquisition unit, a sub-pixel edge detection algorithm based on Gaussian fitting is used on the image acquired by the camera to obtain an edge point set, and the edge straight line is fitted using the edge point set;

[0035] An intersection acquisition unit, which obtains a straight line intersection based on the edge straight line;

[0036] A size acquisition unit, based on the obtained pose transformation matrix, converts the straight line intersection to the world coordinate system, and acquires the size.

[0037] Compared with the prior art, the advantages and beneficial effects of the present application are:

[0038] (1) The camera is calibrated by using a coordinate measuring machine, the coordinate measuring machine has high precision in repeated measurement error, and can ensure the absolute precision in the world coordinate system of the multi-camera; the coordinate measuring range is large, which meets the high-precision large-size measurement, a sufficient number of markers can be marked on the calibration whiteboard to improve the calibration precision, and the installation conditions of the multi-camera are low at the same time;

[0039] (2) The sub-pixel edge detection algorithm based on Gaussian fitting is used to obtain the edge point set, the edge extraction effect is more accurate, and the algorithm runs fast and efficiently, so that the size measurement is obtained with high precision and high efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the description of the embodiments of the present application or the prior art will be briefly introduced. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0041] Figure 1A flowchart of one embodiment of the multi-camera large-size vision measurement method according to the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application.

[0043] Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application. In the description of the present application, it should be understood that the terms “center”, “upper”, “lower”, “front”, “rear”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inner”, “outer” and the like indicate the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0044] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms “mounting”, “connection”, “connecting” should be understood in a broad sense, for example, can be fixed connection, can also be detachable connection, or integrally connected. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0045] The terms “first” and “second” are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with “first” and “second” can explicitly or implicitly include one or more features. In the description of the present application, unless otherwise specified, the meaning of “multiple” is two or more.

[0046] In order to realize high-precision and high-efficiency size measurement of large-size objects (such as pole pieces) under multi-camera, the present application relates to a multi-camera large-size vision measurement method and system, and the multi-camera large-size vision measurement method is realized based on a multi-camera large-size vision measurement system.

[0047] As follows, the multi-camera large-size vision measurement method according to the present application will be described in detail. Figure 1 The vision measurement method will be described.

[0048] S1: Install multiple cameras for large-size measurement, wherein each camera has no overlapping area in the field of view.

[0049] The application relates to size measurement of a large-size object by multiple cameras, the field of view of the multiple cameras has no overlapping area, and the multiple cameras acquire the positions of local areas at two ends of the object to be measured.

[0050] The number of cameras is determined according to the shape and size of the object.

[0051] According to the size of the large-size object (for example, a pole piece) to be measured, the position of the field of view of each camera is determined, the field of view range of the monocular camera is determined according to the camera and the lens type, and the optimal field of view and the clarity and the like are adjusted and fixed.

[0052] In the object measurement under the condition that the fields of view of the multiple cameras have no overlapping area, the pose relationship between the multiple cameras needs to be determined, and the world coordinate system of the world is suggested for the multiple cameras, and the transformation relationship between each camera and the world coordinate system is calculated, that is, the pose transformation matrix (including a rotation matrix R and a translation matrix T) between each camera and the world coordinate system is calculated.

[0053] The corresponding points collected by each camera are converted into the world coordinate system of the world by using the pose transformation matrix, so that the world coordinates of the corresponding points under each camera are obtained.

[0054] Therefore, the corresponding points under the multiple cameras are all converted into the world coordinate system of the world, and the actual size of the large-size object is calculated.

[0055] It is known that the corresponding relationship between the image pixel coordinate system and the world coordinate system is as follows.

[0056]

[0057] Where (u, v) is the image pixel coordinate system, Zc is a scale factor, f / S x , f / S y are the normalized focal lengths on the x-axis and the y-axis respectively, u0 and v0 are the image centers, (X W , Y W , Z W ) is the world coordinate system, K is the camera intrinsic parameter, and R and T are the camera extrinsic parameters (that is, the pose transformation matrix).

[0058] Where the camera intrinsic parameter remains unchanged after the camera type is determined, and the camera extrinsic parameter changes with the change of the camera pose.

[0059] In actual measurement, the camera lens distortion can be eliminated by replacing the telecentric lens to eliminate the influence of the camera lens distortion.

[0060] The camera extrinsic parameter is acquired by using the following three-coordinate calibration method.

[0061] S2: Using a coordinate measuring machine to mark a set of coordinate points in the field of view of each camera on the calibration whiteboard, and record the coordinate positions and order of each coordinate point.

[0062] The calibration whiteboard is used to mark the corresponding points between the coordinate measuring machine and the multiple cameras, and therefore the size of the calibration whiteboard should be able to cover the field of view of each camera and be within the range of the coordinate measuring machine.

[0063] Suppose the number of cameras in this application is four (including the first camera, the second camera, the third camera, and the fourth camera), and the field of view of each camera includes one vertex of the polar slice, and there is no overlapping area in the field of view of each camera.

[0064] Note that the size of the calibration whiteboard is S_w*S_h, the field of view of the installed first camera is R1_w*R1_h, the field of view of the installed second camera is R2_w*R2_h, the field of view of the installed third camera is R3_w*R3_h, and the field of view of the installed fourth camera is R4_w*R4_h, where the sum of R1_w*R1_h, R2_w*R2_h, R3_w*R3_h, and R4_w*R4_h is less than S_w*S_h.

[0065] In use, the calibration whiteboard is placed under the coordinate measuring machine, and the coordinate points are calibrated in the world coordinate system of the coordinate measuring machine.

[0066] In the field of view R1_w*R1_h, the coordinate measuring machine marks the first set of coordinate points corresponding to the first camera, and records the coordinate positions and order of the first set of coordinate points.

[0067] In the field of view R2_w*R2_h, the coordinate measuring machine marks the second set of coordinate points corresponding to the second camera, and records the coordinate positions and order of the second set of coordinate points.

[0068] In the field of view R3_w*R3_h, the coordinate measuring machine marks the third set of coordinate points corresponding to the third camera, and records the coordinate positions and order of the third set of coordinate points.

[0069] In the field of view R4_w*R4_h, the coordinate measuring machine marks the fourth set of coordinate points corresponding to the fourth camera, and records the coordinate positions and order of the fourth set of coordinate points.

[0070] The number of coordinate points in each set is multiple, for example, 15 can be selected.

[0071] Each of the three sets of three-dimensional marker points includes a set of three-dimensional marker points A for calculating a pose transformation matrix and a set of three-dimensional marker points B for verifying the calculated pose transformation matrix, for example, the number of set A is 12, and the number of set B is 3.

[0072] When marking points on the coordinate measuring machine, the three-dimensional positions (Xi, Yi, Zi) of the points can be obtained, and the order of the marked points can be set by controlling the movement of the coordinate measuring machine, for example, from left to right, from top to bottom, etc.

[0073] During the process of marking points on the coordinate measuring machine, the probe of the coordinate measuring machine is movable, but the calibration board is not movable.

[0074] In order to realize the calibration of the calibration points and facilitate the subsequent image acquisition of the camera, the probe of the coordinate measuring machine in the present application adopts a standard small circular probe with a marker color, and the points are marked on the calibration whiteboard in a contact manner, wherein the marker points follow the uniform non-collinear principle.

[0075] As described above, the three-dimensional positions and orders of four sets of three-dimensional marker points corresponding to four cameras can be obtained on the calibration whiteboard.

[0076] As described above, all the three-dimensional marker points are in the world coordinate system of the coordinate measuring machine.

[0077] S3: The camera acquires images of the three-dimensional marker points corresponding to its field of view on the calibration whiteboard, obtains the regions of each three-dimensional marker point, and obtains the region center positions of each region.

[0078] As described above, a plurality of sets of three-dimensional marker points corresponding to the field of view ranges of a plurality of cameras have been marked on the calibration whiteboard.

[0079] Then, the calibration whiteboard is placed under the visual measurement system where the plurality of cameras have been installed, the positions of the calibration whiteboard and the plurality of cameras are adjusted, and it is ensured that the existing plurality of three-dimensional marker points corresponding to the field of view of a single camera can be clearly and fully covered within the field of view of the single camera.

[0080] For each camera, a plurality of three-dimensional marker points within the corresponding field of view range are acquired.

[0081] Since the calibration whiteboard is marked, the acquired points (i.e., marker points) within the field of view of a single camera are a set of discrete points with a single background and obvious contrast, and since a telecentric lens is used, the monocular field of view is small (about 3mm to 5mm), and a single acquired point in the image is a nearly ideal circular spot area.

[0082] Alternatively, according to the shape of the probe of the coordinate measuring machine, other shapes of acquired points can also be obtained.

[0083] It should be noted that the order of the three coordinate marker points is to collect the images of the marker points by the camera in the same order, so as to complete the point set pairing.

[0084] If the area where the three coordinate marker points are located is obtained, for example, the circular spot area involved in the present application, a circular spot extraction method is adopted.

[0085] (1) First, the image collected by the camera can be numerically binarized.

[0086] Since the calibration whiteboard background is single and the marker points are marked with a marker color, image segmentation is achieved by binarization processing.

[0087] Binarization can convert the image into a binary image.

[0088] (2) Secondly, the area where each three coordinate marker point is located is extracted.

[0089] For the binary image, the area where the three coordinate marker points are located is obtained according to the pixel of the gray value, that is, the multiple gray values of the area are obtained.

[0090] (3) Then, the area is circularly fitted to obtain the circular spot area corresponding to each three coordinate marker point.

[0091] The commonly used least square method can be used for circular fitting to obtain a relatively ideal extraction effect.

[0092] (4) Further, the center point position of the circular spot area is calculated.

[0093] The center point position (Ui, Vi) of the circular spot area is the area center position as described above.

[0094] The center point position (Ui, Vi) above is the position in the pixel coordinate system of the camera.

[0095] In this way, the center point positions (Ui, Vi) corresponding to the three coordinate positions (Xi, Yi, Zi) of the plurality of three coordinate marker points in the field of view of the camera can be obtained.

[0096] S4: Using the three coordinate positions of the three coordinate marker points in the field of view and the corresponding area center positions, the pose transformation matrix between the pixel coordinate system and the world coordinate system is calculated.

[0097] According to the transformation relationship between the pixel coordinate system and the world coordinate system as described above, based on the three coordinate positions (Xi, Yi, Zi) and the corresponding center point positions (Ui, Vi) of each camera obtained in S3, the pose transformation matrix can be calculated.

[0098] In this application, the three-dimensional position in the field of view and its corresponding area center position are used to solve the pose transformation matrix by SVD (Singular Value Decomposition).

[0099] SVD is an algorithm widely used in the field of machine learning. The method of solving the pose transformation matrix by SVD in this application will be briefly introduced as follows.

[0100] First, two corresponding point sets are constructed, which are the point set P (p1, p2,..., p n ) corresponding to the pixel coordinate system and the point set Q (q1, q2,..., q n ) corresponding to the world coordinate system.

[0101] The point set Q constructed here refers to a set of three-coordinate marker points A as described above, and the point set P corresponds to the point set A.

[0102] Now we want to calculate the corresponding pose transformation matrix (R, t) according to the data of the two point sets. It can be known that this is actually a least squares optimization problem, and the problem can be described by the following calculation formula.

[0103] (1)

[0104] Where w i > 0 is the weight of each point pair in the point set.

[0105] If we want to find the minimum value of formula (1), that is, to find the solution of formula (1) by taking the derivative of R and t as 0.

[0106] Let R of formula (1) be an invariant, and take the derivative of t, and let F(t)=(R,t), and take the derivative of F(t) to get:

[0107] (2)

[0108] Let (3)

[0109] Substitute it into formula (2) to get (4)

[0110] Substitute formula (4) into formula (1) to get

[0111] (5)

[0112] Where and are equivalent to the decentralization operation of the original point set to get a new point set, and then formula (5) is converted as follows.

[0113] (6)

[0114] Expanding equation (6) in matrix form, we have

[0115]

[0116] = (7)

[0117] It is known that is a 1xd vector, R T is a dxd vector, y i is a dxl vector, thus is a scalar, and we have .

[0118] Therefore, equation (7) is equivalent to (8)

[0119] Equation (6) is transformed to (9)

[0120] Since and are independent of the rotation matrix R, equation (9) is transformed to

[0121] (10)

[0122]

[0123] = (11)

[0124] Equation (11) is denoted as .

[0125] Since tr(BA) = tr(AB), we have

[0126]

[0127] Let S = XWY T , and SVD decomposition of S gives

[0128] Since U, R and V are orthogonal matrices, M = V T RU is also an orthogonal matrix.

[0129] To find the maximum R, we have I = M = V T RU, and R = VU T .

[0130] After obtaining R, we can get t by substituting equation (4).

[0131] A set of three coordinate marker points B as described above is used to verify the rotation matrix R and the translation matrix T calculated as described above.

[0132] For example, three pairs of point sets are used for verification.

[0133] Two corresponding verification point sets are constructed, a point set P'(p 13 ,p 12 ,p 15 ) corresponding to the pixel coordinate system and a point set Q'(q 13 ,q 14 ,q 15 ) corresponding to the world coordinate system.

[0134] The point set Q' constructed here refers to the set of three coordinate marker points B as described above, and the point set P' corresponds to the point set B.

[0135] For example, by comparing the three pairs of point set data Rp 13 +t t , Rp 14 +t t , Rp 15 +t t with q 13 , q 14 and q 15 respectively, when the deviation (for example, the Euclidean distance between them) is within a preset deviation (for example, 5um), it is considered that the calculated R and t meet the subsequent measurement accuracy, and if the deviation is relatively large, it is considered that R and t do not meet the subsequent measurement requirements, and the data point collection and calculation need to be performed again.

[0136] S5: repeatedly performing S3 and S4 until the pose transformation matrix of all cameras is obtained.

[0137] The pose transformation matrix of all cameras is the same, so the pose transformation matrix corresponding to each camera is obtained in the manner described above.

[0138] That is, if there are four cameras, the pose transformation matrix of the first camera includes the rotation matrix R1 and the translation matrix T1, the pose transformation matrix of the second camera includes the rotation matrix R2 and the translation matrix T2, the pose transformation matrix of the third camera includes the rotation matrix R3 and the translation matrix T3, and the pose transformation matrix of the fourth camera includes the rotation matrix R4 and the translation matrix T4.

[0139] Therefore, the positions obtained under each camera can be converted into a unified world coordinate system, and multi-camera calibration is completed.

[0140] Referring to Figure 1 The number of cameras N can be polled until the pose transformation matrix is obtained for all cameras.

[0141] The multi-camera calibration related in the present application adopts a marked whiteboard and uses a coordinate measuring machine to mark the marker points, and has the following technical advantages.

[0142] (1) High measurement accuracy, overcoming the problem of limited accuracy of traditional chessboard calibration method.

[0143] (2) The size of the marked whiteboard is freely made, and a sufficient number of marker points can be marked on the marked whiteboard to improve the calculation accuracy of the camera external parameters, and overcome the problem that the size specification of the traditional calibration board is limited, and it is difficult to use a standard calibration board to establish a unified coordinate system for multi-camera connection for large field of view measurement application.

[0144] (3) The calibration whiteboard related in the present application can be a white paper, which is relatively thin, and overcomes the problem that the existing chessboard calibration board has a certain thickness, which deviates from the best measurement field of view of the telecentric lens of the camera.

[0145] (4) The three-coordinate measuring machine has high repeatability error accuracy, which can be below 5um, can ensure the absolute accuracy of the multi-camera in the unified world coordinate, meet the um-level large-size measurement and detection requirements, and the three-coordinate measuring machine has a large measurement range, which can meet the high-precision size measurement of large-size objects. The corresponding calibration whiteboard can be made into a large-size whiteboard, and during the marking process of the marker points, it is not necessary to move the calibration whiteboard, and the reliability and stability of the marker points marked in the field of view of each camera are determined.

[0146] (5) The best shooting distance of the telecentric lens of each camera is realized to achieve clear collection of three-coordinate marker points, to achieve consistency of the best shooting distance and the best shooting distance during subsequent size measurement, and to improve the size measurement accuracy.

[0147] S6: A sub-pixel edge detection algorithm based on Gaussian fitting is used to obtain an edge point set, and an edge straight line is fitted using the edge point set.

[0148] If the size of a large-size object is to be measured, the positions of the vertices need to be obtained.

[0149] The large-size object related in the present application is a pole piece, which has a relatively regular rectangular shape, and the four top corners are circular arc corners.

[0150] Therefore, the measurement of the size of the pole piece is to obtain four intersection points as measurement points by intersecting the four edge straight lines two by two.

[0151] In order to improve the measurement accuracy, a sub-pixel edge detection algorithm based on Gaussian fitting is used to extract the edge to obtain an edge point set.

[0152] The sub-pixel edge detection algorithm based on Gaussian fitting is a method widely used in the existing vision field. The general principle is as follows: firstly, a series of points near the edge are selected, the gray values of the points are obtained, and then the gradient values of the points are obtained; then, Gaussian curve is used to fit the gradient values of the points; and finally, the position of the symmetry axis of the Gaussian curve obtained through the fitting curve is the sub-pixel position.

[0153] The detection algorithm can well realize sub-pixel positioning, and has shorter running time and higher efficiency.

[0154] In the present application, four cameras are arranged, and the field of view of each camera covers one vertex of the pole piece. Therefore, adjacent cameras can capture images containing the same edge, and therefore the same edge only participates in one calculation.

[0155] For example, the first camera captures the first vertex, the adjacent edge straight lines of the first vertex are L1 and L2, and the intersection point of the straight lines L1 and L2 is P1; the second camera captures the second vertex, the adjacent edge straight lines of the second vertex are L2 and L3, and the intersection point of the straight lines L2 and L3 is P2; the third camera captures the third vertex, the adjacent edge straight lines of the third vertex are L3 and L4, and the intersection point of the straight lines L3 and L4 is P3; and the fourth camera captures the fourth vertex, the adjacent edge straight lines of the fourth vertex are L4 and L1, and the intersection point of the straight lines L4 and L1 is P4.

[0156] The edge point set is obtained by the sub-pixel edge detection algorithm based on Gaussian fitting, and then the edge straight lines L1, L2, L3 and L4 are fitted by using the edge point set (for example, using the least square method based on weight).

[0157] S7: Obtain the intersection point of the straight lines based on the edge straight lines.

[0158] Two straight lines L1 and L2 are represented as y1=k1x+b1 and y2=k2x+b2, respectively. Then, the intersection point P1 of the straight lines L1 and L2 is obtained according to the points (x1, y1) and (x2, y2) on the straight line L1 and the points (x3, y3) and (x4, y5) on the straight line L2.

[0159] Similarly, the other intersection points P2, P4 or P2, P3 are obtained. The reason is that after the intersection points P1 and P2 are obtained, any one of P3 or P4 can obtain the length of the pole piece (i.e., P1P4 or P2P3).

[0160] S8: Convert the intersection points of the straight lines to the world coordinate system based on the pose transformation matrix obtained in S5.

[0161] As described above, the intersection points P1 to P4 are all in the pixel coordinate system, and are based on the pose transformation matrix calculated as above.

[0162] The intersection P1 is converted to the world coordinate system by using the pose transformation matrix (R1, T1) to obtain P1' (X1, Y1, Z1).

[0163] The intersection P2 is converted to the world coordinate system by using the pose transformation matrix (R2, T2) to obtain P2' (X2, Y2, Z2).

[0164] The intersection P3 is converted to the world coordinate system by using the pose transformation matrix (R3, T4) to obtain P3' (X3, Y3, Z3).

[0165] The intersection P4 is converted to the world coordinate system by using the pose transformation matrix (R4, T4) to obtain P4' (X4, Y4, Z4).

[0166] In this way, the positions of all the intersections are in the world coordinate system.

[0167] S9: Based on the converted positions, the dimensions are obtained.

[0168] The width of the pole piece is obtained by using the distance between P1' and P2' (or P3' and P4'), and the length of the pole piece is obtained by using the distance between P1' and P4' (or P2' and P3').

[0169] The large-size object as described above is not limited to the pole piece, and can be other large-size objects whose dimensions are obtained by using the positions of the vertices.

[0170] The multi-camera large-size vision measurement method disclosed in the present application has high universality and strong generalization, and can be popularized to the automobile parts industry, especially the engine and other large-size high-precision detection industry fields.

[0171] The present application also provides a multi-camera large-size vision measurement system, which is used to implement the multi-camera large-size vision measurement system as described above.

[0172] The multi-camera is installed for large-size measurement, and the fields of view of the cameras do not overlap.

[0173] The size of the calibration whiteboard can cover the fields of view of the installed cameras, and is located within the range of the coordinate measuring machine.

[0174] The coordinate measuring machine marks a set of three-coordinate marking points (point set A and point set B as described above) in the fields of view of the cameras on the calibration whiteboard, and records the three-coordinate positions and sequences of the three-coordinate marking points.

[0175] Each camera respectively collects images of the three-coordinate marker points in its field of view on the calibration whiteboard, and the first acquisition unit acquires the regions of the three-coordinate marker points in the images collected by each camera and the region center positions of the regions.

[0176] The calculation unit calculates the pose transformation matrix between the pixel coordinate system of the camera and the unified coordinate system of the three-coordinate measuring machine for each camera by using the three-coordinate positions of the three-coordinate marker points in the field of view and the corresponding region center positions.

[0177] The sub-pixel edge detection algorithm based on Gaussian fitting is used for the images taken by the camera, and the second acquisition unit obtains the edge point set and fits the edge straight line by using the edge point set.

[0178] The third acquisition unit obtains the intersection point of the straight lines based on the straight lines.

[0179] The fourth acquisition unit converts the intersection point of the straight lines to the same coordinate system based on the obtained pose transformation matrix and acquires the size.

[0180] The process based on the specific implementation of each component in the visual measurement system is described above with reference to the specific description in the visual measurement method, and will not be repeated here.

[0181] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-camera large-size visual measurement method, characterized in that, Including the following: S1: Install multiple cameras for large-scale measurements, with no overlapping fields of view among the cameras; S2: Using a coordinate measuring machine, mark a set of coordinate measuring machine points on a calibration whiteboard, each within the field of view of each camera, and record the coordinate positions and order of each coordinate measuring machine point. The size of the calibration whiteboard is sufficient to cover the field of view of each camera that has been installed and is within the range of the coordinate measuring machine. S3: The camera acquires images of the three coordinate markers corresponding to its field of view on the calibration whiteboard, obtains the region where each three coordinate marker is located, and obtains the center position of each region. S4: Using the three coordinate positions of the three coordinate markers within the field of view and their corresponding regional center positions, calculate the pose transformation matrix between the pixel coordinate system of the camera and the world coordinate system of the coordinate measuring machine. S5: Repeat S3 and S4 until the pose transformation matrices of all cameras are obtained; S6: Apply a sub-pixel edge detection algorithm based on Gaussian fitting to the image captured by the camera to obtain an edge point set, and then use the edge point set to fit an edge line; S7: Obtain the intersection point of the straight lines based on the edge lines; S8: Based on the pose transformation matrix obtained in S5, transform the intersection points of the straight lines to the world coordinate system; S9: Obtain the dimensions based on the transformed position.

2. The multi-camera large-size visual measurement method according to claim 1, characterized in that, The coordinate measuring machine uses a standard micro-circular probe with marked colors to mark coordinate points.

3. The multi-camera large-size visual measurement method according to claim 2, characterized in that, In step S3, the camera acquires images of the three-coordinate markers corresponding to its field of view on the calibration whiteboard, and obtains the region where each three-coordinate marker is located, specifically as follows: The image is binarized; Extract the region where each coordinate marker point is located; Perform circular fitting on the area to obtain the circular patch area corresponding to each three coordinate marker point.

4. The multi-camera large-size visual measurement method according to claim 1, characterized in that, The three-coordinate markers recorded in S2 include a first set of markers for calculating the pose transformation matrix and a second set of markers for verifying the calculated pose transformation matrix.

5. The multi-camera large-size visual measurement method according to claim 1, characterized in that, The camera uses a telecentric lens.

6. The multi-camera large-size visual measurement method according to claim 1, characterized in that, In S4, the pose transformation matrix is ​​solved using the three coordinate positions of the three coordinate markers within the field of view and their corresponding region center positions.

7. The multi-camera large-size visual measurement method according to claim 1, characterized in that, When the large size is the same as the large size of the electrode, there must be at least one vertex of the electrode within the field of view of the camera.

8. A multi-camera large-size visual measurement system, characterized in that, include: Multiple cameras are installed for large-scale measurements, with no overlapping fields of view among the cameras; The calibration whiteboard is large enough to cover the field of view of each camera that has been installed. The coordinate measuring machine (CMM) has a calibration whiteboard whose size is within the range of the CMM. The CMM marks a set of coordinate markers on the calibration whiteboard, which are located within the field of view of each camera, and records the coordinate positions and order of each coordinate marker. The first acquisition unit acquires images of the three-coordinate markers on the calibration whiteboard corresponding to their field of view by each camera. For the images acquired by each camera, the unit acquires the region where each three-coordinate marker is located in the image and acquires the center position of the region where each region is located. The calculation unit, for each camera, uses the three-coordinate positions of the three-coordinate marker points within the field of view and their corresponding regional center positions to calculate the pose transformation matrix between the pixel coordinate system of the camera and the world coordinate system of the coordinate measuring machine. The second acquisition unit uses a sub-pixel edge detection algorithm based on Gaussian fitting to obtain an edge point set from the image captured by the camera, and then uses the edge point set to fit an edge line. The third acquisition unit acquires the intersection point of the straight lines based on the edge straight lines; The fourth acquisition unit, based on the acquired pose transformation matrix, transforms the intersection point of the straight lines to the world coordinate system and obtains the dimensions.

Images