Calibration method, multi-camera probe device and flying probe tester
Through the multi-camera calibration method, homogeneous transformation and affine transformation calculate the transformation matrix, the fast and high-precision calibration of the camera and probe is achieved, solving the problem of time-consuming, labor-intensive and low accuracy in the existing technology, and achieving high-precision Camera and probe positioning.
Patent Information
- Application Number
- CN202210088304.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-25
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-01-25
AI Technical Summary
In the prior art, the calibration of external components of the camera is time-consuming and labor-intensive, with low accuracy and cannot meet high-precision positioning application scenarios.
Using multi-camera calibration method, the image of the calibration plate is taken by the first camera, the second camera and the third camera, and the transformation matrix is calculated by homogeneous transformation and affine transformation to determine the mechanical coordinate system and image coordinate system of the camera and probe. The conversion relationship between the two can achieve fast and high-precision calibration.
The calibration process is simplified and the calibration accuracy of the camera and probe is improved. The calibration accuracy of the camera can reach 0.7μm and the calibration accuracy of the probe can reach 3.5μm, achieving one-key calibration, reducing manual teaching steps.
Smart Images

Figure CN114494460B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of machine vision, and in particular to a calibration method, a multi-camera probe device and a flying probe tester. Background Art
[0002] Camera calibration refers to the process of using a camera system to capture an image of a calibration plate, and using the three-dimensional coordinates of known feature points in the calibration plate and the corresponding image coordinates on the image to solve the camera parameters. In the field of machine vision and image measurement, the main equipment for completing measurement and positioning tasks is the camera. In order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, it is necessary to establish a geometric model of camera imaging and accurately calibrate its parameters. Whether in image measurement or machine vision applications, the calibration of camera parameters is a very critical link. The accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera.
[0003] At present, for the calibration of external camera actuators, such as probes, cutting heads, laser inkjet heads, etc., the actual positions are generally taught manually multiple times, and then calibrated by taking pictures with the camera. The calibration is time-consuming and labor-intensive, and the accuracy is very low, which cannot meet the needs of high-precision positioning application scenarios.
[0004] Therefore, in response to the above technical problems, it is necessary to provide a new calibration method. Summary of the invention
[0005] The object of the present invention is to provide a calibration method, a multi-camera probe device and a flying probe tester. The calibration method has a simple calibration implementation process and high calibration accuracy.
[0006] To achieve the above purpose, the technical solution provided by the present invention is as follows:
[0007] In a first aspect, the present invention provides a multi-camera calibration method, which comprises the following steps:
[0008] Establish the image coordinate system XYZ of the first camera p1 and the mechanical coordinate system XYZ m1 , the image coordinate system XYZ of the second camera p2 and the mechanical coordinate system XYZ m2 , and the image coordinate system XYZ of the third camera p3 ;
[0009] Establish the physical coordinate system XYZ of the calibration plate with the preset point on the calibration plate as the origin mb , according to the actual physical distance between the preset point on the calibration plate and the calibration point, the physical coordinates of the calibration point [x mb ,y mb ] T ;
[0010] The first camera, the second camera and the third camera take images of the calibration plate and determine the calibration points in the coordinate system XYZ p1 、XYZ p2 and XYZ p3 The image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T and [x p3 ,y p3 ] T ;
[0011] According to the physical coordinates of the calibration point [x mb ,y mb ] T and the image coordinates of the calibration points [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T ,[x p3 ,y p3 ] T , through homogeneous transformation, we get the coordinate system XYZ mb To coordinate system XYZ p1 、XYZ p2 and XYZ p3 The transformation matrix M mb→p1 、M mb→p2 and M mb→p3 ;
[0012] According to the image coordinates of the calibration point [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T , get the calibration point in the coordinate system XYZ m1 and XYZ m2 The mechanical coordinates in the coordinate system XYZ are obtained through affine transformation. p1 To coordinate system XYZ m1 The transformation matrix M p1→m1 , and coordinate system XYZ p2 To coordinate system XYZ m2 The transformation matrix M p2→m2 ;
[0013] According to the transformation matrix M mb→p1 、M mb→p2 、M mb→p3 、M p1→m1 and Mp2→m2 , get the mechanical coordinate system XYZ of the first camera m1 、The mechanical coordinate system XYZ of the second camera m2 and the image coordinate system XYZ of the third camera p3 The conversion relationship between them.
[0014] In one or more embodiments, the coordinate system XYZ p1 The origin is the image center of the image captured by the first camera, and the coordinate system XYZ p2 The origin is the image center of the image captured by the second camera, and the coordinate system XYZ p3 The origin is the image center of the image taken by the third camera.
[0015] In one or more embodiments, the coordinate system XYZ m1 The origin of the coordinate system XYZ is the mechanical position of the first camera when the first camera photographs the calibration plate. m2 The origin is the mechanical position of the second camera when the second camera photographs the calibration plate.
[0016] In one or more embodiments, the first camera, the second camera, and the third camera are used to capture an image of the calibration plate to determine the calibration point in the coordinate system XTZ. p1 XTZ p2 and XTZ p3 The image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T and [x p3 ,y p3 ] T ,include:
[0017] The calibration plate is photographed by the first camera, the second camera and the third camera to obtain the first image, the second image and the third image of the calibration plate, and the calibration point is determined in the coordinate system XYZ according to the pixel position of the calibration point in the first image, the second image and the third image. p1 、XYZ p2 and XYZ p3 The image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T and [x p3 ,y p3 ] T .
[0018] In one or more embodiments, the homogeneous transformation formula is:
[0019]
[0020] Among them, Z c is the vertical height from the calibration plane of the calibration plate to the plane of the camera's image coordinate system, (u, v) is the point in the calibration plate image acquired by the camera, dX×dY is the physical size of a single pixel, (u0, v0) is the translation from the upper left corner of the calibration plate image acquired by the camera to the origin of the camera's image coordinate system, f is the focal length of the camera lens, (X w ,Y w ,Z w ) is the point coordinate in the physical space, and M is the transformation matrix between the physical coordinate system of the calibration plate and the camera image coordinate system.
[0021] In one or more embodiments, the image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T , get the calibration point in the coordinate system XYZ m1 and XYZ m2 The mechanical coordinates in include:
[0022] According to the image coordinates of the calibration point [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T , calculate the pixel distance between the calibration point and the center of the image, and obtain the physical distance between the calibration point and the center of the first camera and the second camera through the pixel unit. m1 The mechanical coordinates and the center of the second camera in the coordinate system XYZ m2 Get the mechanical coordinates of the calibration point in the coordinate system XYZ m1 and XYZ m2 The mechanical coordinates in .
[0023] In one or more embodiments, the transformation matrix M mb→p1 、M mb→p2 、M mb→p3 、M p1→m1 and M p2→m2 , get the mechanical coordinate system XYZ of the first camera m1 、The mechanical coordinate system XYZ of the second camera m2 and the image coordinate system XYZ of the third camera p3 The conversion relationship between them includes:
[0024] Set coordinate system XYZ mb The middle point is (x mb1 ,y mb1 ), which is in the coordinate system XYZ m1 The coordinates in (x m11 ,y m11 ), which is in the coordinate system XYZ m2 The coordinates in (x m21 ,y m21 ), which is in the coordinate system XYZ p3 The coordinates in (x p31 ,y p31 ), then
[0025]
[0026] According to formula (1), the coordinate system XYZ m2 To coordinate system XYZ m1 The projection matrix is M mb→p1 ·M p1→m1 ·M p2→m2 -1 ·M mb→p2 -1 ; Coordinate system XYZ m1 To coordinate system XYZ m2 The projection matrix is M mb→p2 ·M p2→m2 ·M p1→m1 -1 ·M mb→p1 -1 ; Coordinate system XYZ p3 To coordinate system XYZ m1 The projection matrix is M mb→p1 ·M p1→m1 ·M mb→p3 -1 ; Coordinate system XYZ p3 To coordinate system XYZ m2 The projection matrix is M mb→p2 ·M p2→m2 ·M mb→p3 -1 .
[0027] In one or more embodiments, the preset points and calibration points on the calibration plate are set to be visible on both sides, the first camera and the second camera photograph the preset points and the calibration points through the first side of the calibration plate, and the third camera photographs the preset points and the calibration points through the second side of the calibration plate.
[0028] In one or more embodiments, the calibration plate is provided with a physical coordinate system XYZ for determining the position of the preset point and for determining the calibration plate.mb direction indicator.
[0029] In a second aspect, the present invention provides a probe calibration method, which is applied to a flying probe tester, the flying probe tester comprising a first test axis, a second test axis and a third camera, the first test axis being provided with a first camera and a first probe, the second test axis being provided with a second camera and a second probe, the probe calibration method comprising the following steps:
[0030] Based on the multi-camera calibration method as described above, the calibration of the first camera, the second camera and the third camera is completed;
[0031] Move the first probe to the field of view of the third camera, and use the third camera to capture the image of the first probe to obtain the coordinate system XYZ of the first probe tip. p3 The coordinates (x p1针 ,y p1针 ,1), according to the coordinate system XYZ p3 and coordinate system XYZ m1 The conversion relationship between the first probe tip and the coordinate system XYZ is obtained. m1 The coordinates (x m1针 ,y m1针 ,1) T ;
[0032] According to the mechanical position (x m1c ,y m1c ), and the mechanical position of the first test axis when the third camera photographs the first probe (x 1针 ,y 1针 ,z 1针 ), obtain the actual offset (Δx1, Δy1) from the tip of the first probe to the center of the first camera, and complete the calibration of the first probe to the first camera;
[0033] Move the second probe to the field of view of the third camera, and use the third camera to capture the image of the second probe to obtain the position of the second probe tip in the coordinate system XYZ. p3 The coordinates (x p2针 ,y p2针 ,1), according to the coordinate system XYZ p3 and coordinate system XYZ m2 The conversion relationship between the two is obtained to obtain the coordinate system XYZ of the second probe tip. m2 The coordinates (x m2针 ,y m2针 ,1) T ;
[0034] According to the mechanical position (x m2c ,ym2c ), and the mechanical position of the second test axis when the third camera photographs the second probe (x 2针 ,y 2针 ,z 2针 ), and obtain the actual offset (Δx2, Δy2) from the tip of the second probe to the center of the second camera, completing the calibration of the second probe to the second camera.
[0035] In one or more embodiments, the first probe tip is in the coordinate system XYZ m1 The coordinates (x m1针 ,y m1针 ,1) T for:
[0036] (x m1针 ,y m1针 ,1) T =M mb→p1 ·M p1→m1 ·M mb→p3 -1 ·(x p1针 ,y p1针 ,1) T .
[0037] In one or more embodiments, the actual offset (Δx1, Δy1) from the first probe tip to the center of the first camera is:
[0038] (Δx1,Δy1)=(x 1针 -x m1c +x m1针 ,y 1针 -y m1c +y m1针 ).
[0039] In one or more embodiments, the second probe tip is in the coordinate system XYZ m2 The coordinates (x m2针 ,y m2针 ,1) T for:
[0040] (x m2针 ,y m2针 ,1) T =M mb→p2 ·M p2→m2 ·M mb→p3 -1 ·(x p2针 ,y p2针 ,1) T .
[0041] In one or more embodiments, the actual offset (Δx2, Δy2) from the second probe tip to the center of the second camera is:
[0042] (Δx2,Δy2)=(x 2针 -x m2c +x m2针 ,y 2针 -y m2c +y m2针 ).
[0043] In the third aspect, the present invention provides a multi-camera probe device for the aforementioned probe calibration method, comprising: a base frame, a first drive module, a second drive module, a first test axis, a second test axis and a third camera; the first drive module and the second drive module are arranged on the base frame, the first test axis is installed on the first drive module, and the second test axis is installed on the second drive module; the first test axis comprises a first bracket, a first camera and a first probe, the first bracket is installed on the first drive module, and the first camera and the first probe are installed on the first bracket; the second test axis comprises a second bracket, a second camera and a second probe, the second bracket is installed on the second drive module, and the second camera and the second probe are installed on the second bracket.
[0044] In one or more embodiments, a calibration plate fixing member is further provided on the base frame, and the third camera is fixed below the calibration plate fixing member.
[0045] In a fourth aspect, the present invention provides a flying probe tester, which includes a memory, a processor and the aforementioned multi-camera probe device, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the aforementioned probe calibration method.
[0046] Compared with the prior art, the calibration method, multi-camera probe device and flying probe tester provided by the present invention have a simple calibration process for the camera system and high calibration accuracy. After completing the calibration of the camera system, the probe can be calibrated with one key without the need to manually teach the actual position multiple times. After replacing the probe, there is no need to manually teach the probe position again. By taking visual photos, the actual position of the new probe in the system can be directly determined based on the previous calibration. The calibration accuracy of the camera can reach less than 0.7μm, and the calibration accuracy of the probe can reach less than 3.5μm. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 is a schematic diagram of the three-dimensional structure of a multi-camera probe device in one embodiment of the present invention;
[0048] Figure 2 is a flow chart of a multi-camera calibration method in one embodiment of the present invention;
[0049] Figure 3is a flow chart of a probe calibration method in one embodiment of the present invention;
[0050] Figure 4 1 is a structural block diagram of a flying probe tester in one embodiment of the present invention.
[0051] Description of main reference numerals:
[0052] 1-multi-camera probe device, 11-base frame, 12-first driving module, 13-second driving module, 14-first test axis, 15-second test axis, 16-third camera, 17-calibration plate fixing part, 121-first moving axis mechanism, 122-first transfer mechanism, 131-second moving axis mechanism, 132-second transfer mechanism, 141-first bracket, 142-first camera, 143-first probe, 151-second bracket, 152-second camera, 153-second probe. DETAILED DESCRIPTION
[0053] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.
[0054] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.
[0055] Please refer to Figure 1 As shown, a multi-camera probe device 1 in one embodiment of the present invention can be applied to a flying probe tester. The multi-camera probe device 1 includes: a base frame 11, a first driving module 12, a second driving module 13, a first test axis 14, a second test axis 15 and a third camera 16.
[0056] In an exemplary embodiment, the base frame 11 is configured to be substantially in the shape of a rectangular parallelepiped, and the first driving module 12 and the second driving module 13 are mounted on the top of the base frame 11. The first test shaft 14 is mounted on the first driving module 12, and the first test shaft 14 can move under the drive of the first driving module 12. The second test shaft 15 is mounted on the second driving module 13, and the second test shaft 15 can move under the drive of the second driving module 13.
[0057] Specifically, the first driving module 12 includes a first moving axis mechanism 121 arranged along the x-axis direction and a first transfer mechanism 122 arranged along the y-axis direction. The first moving axis mechanism 121 is installed on the top of the base frame 11, the first transfer mechanism 122 is installed on the first moving axis mechanism 121, and the first test axis 14 is installed on the first transfer mechanism 122; and the first transfer mechanism 122 can move along the x-axis direction under the drive of the first moving axis mechanism 121, and the first test axis 14 can move along the y-axis direction under the drive of the first transfer mechanism 122.
[0058] Specifically, the second driving module 13 includes a second moving axis mechanism 131 arranged along the x-axis direction and a second transfer mechanism 132 arranged along the y-axis direction. The second moving axis mechanism 131 is installed on the top of the base frame 11, the second transfer mechanism 132 is installed on the second moving axis mechanism 131, and the second test axis 15 is installed on the second transfer mechanism 132; and the second transfer mechanism 132 can move along the x-axis direction under the drive of the second moving axis mechanism 131, and the second test axis 15 can move along the y-axis direction under the drive of the second transfer mechanism 132.
[0059] The first driving module 12 and the second driving module 13 are independent of each other. The first test shaft 14 and the second test shaft 15 can be driven by the first driving module 12 and the second driving module 13 to move independently.
[0060] Specifically, the first test axis 14 includes a first bracket 141, a first camera 142 and a first probe 143. The first bracket 141 is mounted on the first transfer mechanism 122 of the first driving module 12, and the first camera 142 and the first probe 143 are mounted on the first bracket 141. Driven by the first driving module 12, the first camera 142 and the first probe 143 can move synchronously.
[0061] Specifically, the second test axis 15 includes a second bracket 151, a second camera and a second probe. The second bracket 151 is mounted on the first transfer mechanism 122 of the second drive module 13, and the second camera 152 and the second probe are mounted on the second bracket 151. Driven by the second drive module 13, the second camera 152 and the second probe 153 can move synchronously.
[0062] In an exemplary embodiment, a calibration plate fixing member 17 is further provided on the base frame 11. When performing camera calibration, the calibration plate can be fixed on the calibration plate fixing member 17, so that the camera can shoot the calibration plate and obtain image information of the calibration plate.
[0063] Specifically, the third camera 16 is fixed under the calibration plate fixture 17. In the multi-camera probe device 1, the third camera 16 is fixedly arranged, so during the calibration process, the camera center of the third camera 16 is fixed, that is, the image coordinate system established based on the third camera 16 is also fixed, and there is no position offset.
[0064] Specifically, the first camera 142 , the second camera 152 and the third camera 16 are also provided with a light source, which can provide illumination when the first camera 142 , the second camera 152 and the third camera 16 capture images, so that the cameras can capture clearer images.
[0065] Please refer to Figure 2 FIG. 1 is a flow chart of a multi-camera calibration method in one embodiment of the present invention, which can be used for the aforementioned multi-camera probe device 1. The multi-camera calibration method comprises the following steps:
[0066] S101: Establishing the image coordinate system XYZ of the first camera 142 p1 and the mechanical coordinate system XYZ m1 , the image coordinate system XYZ of the second camera 152 p2 and the mechanical coordinate system XYZ m2 , and the image coordinate system XYZ of the third camera 16 p3 .
[0067] Specifically, the image coordinate system XYZ of the first camera 142 p1 , is a coordinate system established based on the image captured by the first camera 142, the coordinate system XYZ p1 The origin of is the image center of the image captured by the first camera 142. For example, a plane perpendicular to the shooting direction (lens orientation) of the first camera 142 and passing through the center of the first camera 142 can be used as the coordinate system XYZ p1 At this time, the center of the first camera 142 is the image center of the image captured by the first camera 142, that is, the coordinate system XYZ p1 The origin of .
[0068] Image coordinate system XYZ of the second camera 152 p2 , is a coordinate system established based on the image captured by the second camera 152, the coordinate system XYZ p2 The origin of the coordinate system XYZ is the image center of the image captured by the second camera 152. For example, a plane perpendicular to the shooting direction (lens orientation) of the second camera 152 and passing through the center of the second camera 152 can be used as the coordinate system XYZ p2 At this time, the center of the second camera 152 is the center of the image captured by the first camera, that is, the coordinate system XYZ p2 The origin of .
[0069] Image coordinate system XYZ of the third camera 16 p3 , is a coordinate system established based on the image taken by the third camera 16, the coordinate system XYZ p3 The origin of is the image center of the image captured by the third camera 16. For example, a plane perpendicular to the shooting direction (lens orientation) of the third camera 16 and passing through the center of the third camera 16 can be used as the coordinate system XYZ p3 At this time, the center of the third camera 16 is the image center of the image captured by the third camera 16, that is, the coordinate system XYZ p3 The origin of .
[0070] In other embodiments, the coordinate system XYZ p1 、XYZ p2 and XYZ p3 You can select the origin of the image or other points on the image. You can set it according to actual needs.
[0071] Specifically, the mechanical coordinate system XYZ of the first camera 142 m1 The origin is the mechanical position of the first camera 142 when the first camera 142 photographs the calibration plate. The mechanical position can be determined by the position (x m1c ,y m1c ) to determine.
[0072] Mechanical coordinate system XYZ of the second camera 152 m2 The origin is the mechanical position of the second camera 152 when the second camera 152 photographs the calibration plate. The mechanical position can be determined by the position (x m2c ,y m2c ) to determine.
[0073] S102: Establish the physical coordinate system XYZ of the calibration plate with the preset point on the calibration plate as the origin mb , according to the actual physical distance between the preset point on the calibration plate and the calibration point, the physical coordinates of the calibration point [x mb ,y mb ] T .
[0074] When establishing the physical coordinate system XYZ of the calibration plate mb Before doing so, the calibration plate needs to be positioned on the calibration plate fixing member 17 of the multi-camera probe device 1 .
[0075] Specifically, the physical coordinate system XYZ of the calibration plate mb It can be constructed from the calibration points captured by the camera, and any preset point in the field of view obtained by the camera can be selected as the coordinate system XYZ mbThe origin of the calibration plane of the calibration plate can be used as the coordinate system XYZ mb XY plane.
[0076] Both the preset point and the calibration point are in the coordinate system XYZ mb The XY plane of the calibration point can be calculated based on the actual physical distance between the preset point and the calibration point. mb The physical coordinates [x mb ,y mb ] T .
[0077] S103: The first camera 142, the second camera 152 and the third camera 16 are used to capture an image of the calibration plate and determine the calibration point in the coordinate system XYZ. p1 、XYZ p2 and XYZ p3 The image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T and [x p3 ,y p3 ] T .
[0078] Specifically, the calibration plate is photographed by the first camera 142, the second camera 152 and the third camera 16, and a first image of the calibration plate based on the first camera 142, a second image based on the second camera and a third image based on the third camera 16 are obtained. According to the pixel positions of the calibration points in the first image, the second image and the third image, the pixel position relationship of the calibration points in the first image, the second image and the third image relative to the center of each image (the origin of the image coordinate system of each camera) is calculated to determine the calibration point coordinate system XYZ p1 、XYZ p2 and XYZ p3 The image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T and [x p3 ,y p3 ] T .
[0079] Specifically, the preset points and calibration points on the calibration board are set to be visible on both sides. The first camera 142 and the second camera 152 can shoot the preset points and calibration points of the calibration board through the first side (top side) of the calibration board, and the third camera 16 can shoot the preset points and calibration points of the calibration board through the second side (bottom side) of the calibration board. The calibration board is also provided with a position for determining the preset points and a physical coordinate system XYZ for determining the calibration board. mb Through this identification combined with image analysis, the first camera 142, the second camera 152 and the third camera 16 can automatically identify the position of the preset point and the coordinate system XYZ mb direction, so that the same calibration point on the calibration plate is corresponded in the first image, the second image and the third image.
[0080] S104: According to the physical coordinates [x mb ,y mb ] T and the image coordinates of the calibration points [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T ,[x p3 ,y p3 ] t , through homogeneous transformation, we get the coordinate system XYZ mb To coordinate system XYZ p1 、XYZ p2 and XYZ p3 The transformation matrix M mb→p1 、M mb→p2 and M mb→p3 .
[0081] Specifically, the homogeneous transformation formula is:
[0082]
[0083] Among them, Z c is the calibration plane of the calibration plate (coordinate system XYZ mb XY plane) to the camera's image coordinate plane (coordinate system XYZ p1 、XYZ p2 and XYZ p3 The vertical height of the XY plane), (u, v) is the point in the calibration plate image obtained by the camera, dX×dY is the physical size of a single pixel, (u0, v0) is the translation from the upper left corner of the calibration plate image obtained by the camera to the origin of the camera image coordinate system (camera center), f is the focal length of the camera lens, (X w ,Y w ,Z w) is the point coordinate in the physical space, and M is the transformation matrix between the physical coordinate system of the calibration plate and the camera image coordinate system.
[0084] S105: Based on the image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T , get the calibration point in the coordinate system XYZ m1 and XYZ m2 The mechanical coordinates in the coordinate system XYZ are obtained through affine transformation. p1 To coordinate system XYZ m1 The transformation matrix M p1→m1 , and coordinate system XYZ p2 To coordinate system XYZ m2 The transformation matrix M p2→m2 .
[0085] Specifically, according to the image coordinates of the calibration point [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T , calculate the pixel distance between the calibration point and the image center (corresponding to the camera center), obtain the physical distance between the calibration point and the center of the first camera 142 and the second camera 152 through the pixel unit, and calculate the pixel distance between the calibration point and the center of the first camera 142 and the second camera 152 according to the coordinate system XYZ m1 The mechanical coordinates in the coordinate system XYZ and the center of the second camera 152 m2 The mechanical coordinates in the XYZ coordinate system can be obtained by m1 and XYZ m2 The mechanical coordinates in .
[0086] Based on the image coordinates of the calibration points [x p1 ,y p1 ] T and [x p2 ,y p2 ] T , and the calibration point in the coordinate system XYZ m1 and XYZ m2 The mechanical coordinates in the coordinate system XYZ can be obtained through affine transformation p1 To coordinate system XYZ m1 The transformation matrix M p1→m1 , and coordinate system XYZ p2 To coordinate system XYZ m2 The transformation matrix M p2→m2 .
[0087] S106: According to the transformation matrix Mmb→p1 、M mb→p2 、M mb→p3 、M p1→m1 and M p2→m2 , get the mechanical coordinate system XYZ of the first camera 142 m1 , the mechanical coordinate system XYZ of the second camera 152 m2 and the image coordinate system XYZ of the third camera 16 p3 The conversion relationship between them.
[0088] Specifically, we get the transformation matrix M mb→p1 、M mb→p2 、M mb→p3 、M p1→m1 and M p2→m2 After that, the mechanical coordinate system XYZ of the first camera 142 can be obtained through the common coordinate system between the transformation matrices. m1 , the mechanical coordinate system XYZ of the second camera 152 m2 and the image coordinate system XYZ of the third camera 16 p3 The conversion relationship between them. The process is as follows:
[0089] Set coordinate system XYZ mb The middle point is (x mb1 ,y mb1 ), which is in the coordinate system XYZ m1 The coordinates in (x m11 ,y m11 ), which is in the coordinate system XYZ m2 The coordinates in (x m21 ,y m21 ), which is in the coordinate system XYZ p3 The coordinates in (x p31 ,y p31 ), then
[0090]
[0091] From the above formula, we can get the coordinate system XYZ m2 To coordinate system XYZ m1 The projection matrix is M mb→p1 ·M p1→m1 ·M p2→m2 -1 ·M mb→p2 -1 ; Coordinate system XYZ m1 To coordinate system XYZ m2 The projection matrix is M mb→p2 ·M p2→m2 ·M p1→m1 -1 ·M mb→p1 -1 ; Coordinate system XYZp3 To coordinate system XYZ m1 The projection matrix is M mb→p1 ·M p1→m1 ·M mb→p3 -1 ; Coordinate system XYZ p3 To coordinate system XYZ m2 The projection matrix is M mb→p2 ·M p2→m2 ·M mb→p3 -1 , that is, the calibration of the camera system is completed.
[0092] Based on the above-mentioned multi-camera calibration method, the present invention also provides a probe calibration method. Figure 3 FIG. 1 is a flow chart of a probe calibration method in one embodiment of the present invention. The probe calibration method can be applied to a flying probe tester having the aforementioned multi-camera probe device 1. The multi-camera calibration method comprises the following steps:
[0093] S201: Based on the multi-camera calibration method, complete the calibration of the first camera 142, the second camera 152 and the third camera 16.
[0094] Specifically, after the calibration of the camera is completed, the calibration plate positioned on the calibration plate fixing member 17 needs to be removed to prevent the calibration plate from affecting the subsequent photographing of the probe by the third camera 16 .
[0095] S202: Move the first probe 143 to the field of view of the third camera 16, and use the third camera 16 to capture an image of the first probe 143 to obtain the tip of the first probe 143 in the coordinate system XYZ. p3 The coordinates in the XYZ coordinate system p3 and coordinate system XYZ m1 The conversion relationship between the first probe tip 143 and the coordinate system XYZ is obtained. m1 The coordinates in .
[0096] Specifically, after the third camera 16 acquires the image of the first probe 143, the position of the tip of the first probe 143 in the coordinate system XYZ can be obtained through BLOB analysis. p3 The coordinates (x p1针 ,y p1针 ,1).
[0097] According to the coordinate system XYZ p3 and coordinate system XYZ m1 The conversion relationship between the first probe tip 143 and the coordinate system XYZ is obtained. m1 The coordinates (x m1针 ,y m1针 ,1) T Specifically:
[0098] (x m1针 ,y m1针 ,1) T =M mb→p1 ·M p1→m1 ·M mb→p3 -1 ·(x p1针 ,y p1针 ,1) T .
[0099] S203: According to the mechanical position of the first test axis 14 when the first camera 142 photographs the calibration plate, and the mechanical position of the first test axis 14 when the third camera 16 photographs the first probe 143, the actual offset from the tip of the first probe 143 to the center of the first camera 142 is obtained, and the calibration of the first probe 143 to the first camera 142 is completed.
[0100] Specifically, when the first camera 142 photographs the calibration plate, the mechanical position of the first test axis 14 is (x m1c ,y m1c ), the mechanical position of the first test axis 14 when the third camera 16 photographs the first probe 143 is (x 1针 ,y 1针 ,z 1针 ), then the deviation between the calibrated position of the first probe 143 and the calibrated position of the first camera 142 is (x 1针 -x m1c ,y 1针 -y m1c ).
[0101] Then, according to the first probe 143 tip in the coordinate system XYZ m1 The coordinates (x m1针 ,y m1针 ,1) T , the actual offset from the tip of the first probe 143 to the center of the first camera 142 can be obtained as (Δx1, Δy1) = (x 1针 -x m1c +x m1针 ,y 1针 -y m1c +y m1针 ), that is, the calibration of the first probe 143 to the center of the first camera 142 is completed.
[0102] S204: Move the second probe 153 to the field of view of the third camera 16, and use the third camera 16 to capture an image of the second probe 153 to obtain the tip of the second probe 153 in the coordinate system XYZ. p3 The coordinates in the XYZ coordinate system p3 and coordinate system XYZ m2The conversion relationship between the two is obtained to obtain the position of the tip of the second probe 153 in the coordinate system XYZ m2 The coordinates in .
[0103] Specifically, after the third camera 16 acquires the image of the second probe 153, the position of the tip of the second probe 153 in the coordinate system XYZ can be obtained through BLOB analysis. p3 The coordinates (x p2针 ,y p2针 ,1).
[0104] According to the coordinate system XYZ p3 and coordinate system XYZ m2 The conversion relationship between the first probe tip 143 and the coordinate system XYZ is obtained. m2 The coordinates (x m2针 ,y m2针 ,1) T Specifically:
[0105] (x m2针 ,y m2针 ,1) T =M mb→p2 ·M p2→m2 ·M mb→p3- 1 ·(x p2针 ,y p2针 ,1) T .
[0106] S205: According to the mechanical position of the second test axis 15 when the second camera 152 photographs the calibration plate, and the mechanical position of the second test axis 15 when the third camera 16 photographs the second probe 153, the actual offset from the tip of the second probe 153 to the center of the second camera 152 is obtained, and the calibration of the second camera 152 by the second probe 153 is completed.
[0107] Specifically, when the second camera 152 photographs the calibration plate, the mechanical position of the second test axis 15 is (x m2c ,y m2c ), the mechanical position of the second test axis when the third camera 16 photographs the second probe 153 is (x 2针 ,y 2针 ,z 2针 ), then the deviation between the calibrated position of the second probe 153 and the calibrated position of the second camera 152 is (x 2针 -x m2c ,y 2针 -y m2c ).
[0108] Then, according to the second probe 153 tip in the coordinate system XYZ m2 The coordinates (x m2针 ,ym2针 ,1) T , we can get the actual offset from the tip of the second probe 153 to the center of the second camera 152 (Δx2, Δy2) = (x 2针 -x m2c +x m2针 ,y 2针 -y m2c +y m2针 ), that is, the calibration of the second probe 153 to the center of the second camera 152 is completed.
[0109] like Figure 4 As shown, one embodiment of the present invention also provides a flying probe tester 2, which includes a processor 21, a memory 22 (such as a non-volatile memory), a memory 23, a communication interface 24 and the multi-camera probe device 1 as described above, and the processor 21, the memory 22, the memory 23 and the communication interface 24 are connected together via a bus 25. The processor 21 is used to call at least one program instruction stored or encoded in the memory 22, so that the processor 21 executes the various operating steps and functions of the probe calibration method described in the various embodiments of this specification. When the processor 21 executes the probe calibration method described in the various embodiments of this specification, the specific operating steps can be implemented through the multi-camera probe device 1.
[0110] The memory 22 can be any available medium or data storage device that can be accessed by the computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD), etc.).
[0111] In summary, the calibration method, multi-camera probe device and flying probe tester provided by the present invention have a simple calibration process for the camera system and high calibration accuracy. After completing the calibration of the camera system, the probe can be calibrated with one key without the need to manually teach the actual position multiple times. After replacing the probe, there is no need to manually teach the probe position again. By taking visual photos, the actual position of the new probe in the system can be directly determined based on the previous calibration. The calibration accuracy of the camera can reach less than 0.7μm, and the calibration accuracy of the probe can reach less than 3.5μm.
[0112] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.
Claims
1. A multi-camera calibration method, characterized in that: The following steps are involved: Establish the image coordinate system XYZ of the first camera p1 and the mechanical coordinate system XYZ m1 , the image coordinate system XYZ of the second camera p2 and the mechanical coordinate system XYZ m2 , and the image coordinate system XYZ of the third camera p3 ; Establish the physical coordinate system XYZ of the calibration plate with the preset point on the calibration plate as the origin mb , according to the actual physical distance between the preset point on the calibration plate and the calibration point, the physical coordinates of the calibration point [x mb ,y mb ] T ; The first camera, the second camera and the third camera take images of the calibration plate and determine the calibration points in the coordinate system XYZ p1 、XYZ p2 and XYZ p3 The image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T and [x p3 ,y p3 ] T ; According to the physical coordinates of the calibration point [x mb ,y mb ] T and the image coordinates of the calibration points [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T ,[x p3 ,y p3 ] T , through homogeneous transformation, we get the coordinate system XYZ mb To coordinate system XYZ p1 、XYZ p2 and XYZ p3 The transformation matrix M mb→p1 、M mb→p2 and M mb→p3 ; According to the image coordinates of the calibration point [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T , get the calibration point in the coordinate system XYZ m1 and XYZ m2 The mechanical coordinates in the coordinate system XYZ are obtained through affine transformation. p1 To coordinate system XYZ m1 The transformation matrix M p1→m1 , and coordinate system XYZ p2 To coordinate system XYZ m2 The transformation matrix M p2→m2 ; According to the transformation matrix M mb→p1 、M mb→p2 、M mb→p3 、M p1→m1 and M p2→m2 , get the mechanical coordinate system XYZ of the first camera m1 、The mechanical coordinate system XYZ of the second camera m2 and the image coordinate system XYZ of the third camera p3 The conversion relationship between them; Wherein, the transformation matrix M mb→p1 、M mb→p2 、M mb→p3 、M p1→m1 and M p2→m2 , get the mechanical coordinate system XYZ of the first camera m1 、The mechanical coordinate system XYZ of the second camera m2 and the image coordinate system XYZ of the third camera p3 The conversion relationship between them includes: Set coordinate system XYZ mb The middle point is (x mb1 ,y mb1 ), which is in the coordinate system XYZ m1 The coordinates in (x m11 ,y m11 ), which is in the coordinate system XYZ m2 The coordinates in (x m21 ,y m21 ), which is in the coordinate system XYZ p3 The coordinates in (x p31 ,y p31 ), then According to formula (1), the coordinate system XYZ m2 To coordinate system XYZ m1 The projection matrix is M mb→p1 ·M p1→m1 ·M p2→m2 -1 ·M mb→p2 -1 ; Coordinate system XYZ m1 To coordinate system XYZ m2 The projection matrix is M mb→p2 ·M p2→m2 ·M p1→m1 -1 ·M mb→p1 -1 ; Coordinate system XYZ p3 To coordinate system XYZ m1 The projection matrix is M mb→p1 ·M p1→m1 ·M mb→p3 -1 ; Coordinate system XYZ p3 To coordinate system XYZ m2 The projection matrix is M mb→p2 ·M p2→m2 ·M mb→p3 -1 .
2. The multi-camera calibration method according to claim 1, characterized in that: The coordinate system XYZ p1 The origin is the image center of the image captured by the first camera, and the coordinate system XYZ p2 The origin is the image center of the image captured by the second camera, and the coordinate system XYZ p3 The origin is the image center of the image taken by the third camera.
3. The multi-camera calibration method according to claim 1, characterized in that: The coordinate system XYZ m1 The origin of the coordinate system XYZ is the mechanical position of the first camera when the first camera photographs the calibration plate. m2 The origin is the mechanical position of the second camera when the second camera photographs the calibration plate.
4. The multi-camera calibration method according to claim 1, characterized in that: The first camera, the second camera and the third camera are used to take images of the calibration plate to determine the calibration points in the coordinate system XYZ p1 、XYZ p2 and XYZ p3 The image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T and [x p3 ,y p3 ] T ,include: The calibration plate is photographed by the first camera, the second camera and the third camera to obtain the first image, the second image and the third image of the calibration plate, and the calibration point is determined in the coordinate system XYZ according to the pixel position of the calibration point in the first image, the second image and the third image. p1 、XYZ p2 and XYZ p3 The image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T and [x p3 ,y p3 ] T .
5. The multi-camera calibration method according to claim 1, characterized in that: The homogeneous transformation formula is: Among them, Z c is the vertical height from the calibration plane of the calibration plate to the plane of the camera's image coordinate system, (u, v) is the point in the calibration plate image acquired by the camera, dX×dY is the physical size of a single pixel, (u0, v0) is the translation from the upper left corner of the calibration plate image acquired by the camera to the origin of the camera's image coordinate system, f is the focal length of the camera lens, (X w , Y w , Z w ) is the point coordinate in the physical space, and M is the transformation matrix between the physical coordinate system of the calibration plate and the camera image coordinate system.
6. The multi-camera calibration method according to claim 1, characterized in that: According to the image coordinates [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T , get the calibration point in the coordinate system XYZ m1 and XYZ m2 The mechanical coordinates in include: According to the image coordinates of the calibration point [x p1 ,y p1 ] T ,[x p2 ,y p2 ] T , calculate the pixel distance between the calibration point and the center of the image, and obtain the physical distance between the calibration point and the center of the first camera and the second camera through the pixel unit. m1 The mechanical coordinates and the center of the second camera in the coordinate system XYZ m2 Get the mechanical coordinates of the calibration point in the coordinate system XYZ m1 and XYZ m2 The mechanical coordinates in .
7. The multi-camera calibration method according to claim 1, characterized in that: The preset points and calibration points on the calibration plate are set to be visible on both sides, the first camera and the second camera photograph the preset points and the calibration points through the first side of the calibration plate, and the third camera photographs the preset points and the calibration points through the second side of the calibration plate.
8. The multi-camera calibration method according to claim 7, characterized in that: The calibration plate is provided with a physical coordinate system XYZ for determining the position of the preset point and for determining the calibration plate. mb direction indicator.
9. A probe calibration method, applied to a flying probe tester, the flying probe tester comprising a first test axis, a second test axis and a third camera, the first test axis being provided with a first camera and a first probe, the second test axis being provided with a second camera and a second probe, characterized in that: The probe calibration method comprises the following steps: Based on the multi-camera calibration method described in any one of claims 1 to 8, the calibration of the first camera, the second camera and the third camera is completed; Move the first probe to the field of view of the third camera, and use the third camera to capture the image of the first probe to obtain the coordinate system XYZ of the first probe tip. p3 The coordinates (x p1针 ,y p1针 , 1), according to the coordinate system XYZ p3 and coordinate system XYZ m1 The conversion relationship between the first probe tip and the coordinate system XYZ is obtained. m1 The coordinates (x m1针 ,y m1针 , 1) T ; According to the mechanical position (x m1c ,y m1c ), and the mechanical position of the first test axis when the third camera photographs the first probe (x 1针 ,y 1针 , z 1针 ), obtain the actual offset (Δx1, Δy1) from the tip of the first probe to the center of the first camera, and complete the calibration of the first probe to the first camera; Move the second probe to the field of view of the third camera, and use the third camera to capture the image of the second probe to obtain the position of the second probe tip in the coordinate system XYZ. p3 The coordinates (x p2针 ,y p2针 , 1), according to the coordinate system XYZ p3 and coordinate system XYZ m2 The conversion relationship between the two is obtained to obtain the coordinate system XYZ of the second probe tip. m2 The coordinates (x m2针 ,y m2针 , 1) T ; According to the mechanical position (x m2c ,y m2c ), and the mechanical position of the second test axis when the third camera photographs the second probe (x 2针 ,y 2针 , z 2针 ), obtain the actual offset (Δx2, Δy2) from the tip of the second probe to the center of the second camera, and complete the calibration of the second probe to the second camera; wherein, (x m1针 ,y m1针 ,1) T =M mb→p1 ·M p1→m1 ·M mb→p3 -1 -(x p1针 ,y p1针 ,1) T (Δx1,Δy1)=(x 1针 -x m1c +x m1针 ,y 1针 -y m1c +y m1针 ) (x m2针 ,y m2针 ,1) T =M mb→p2 ·M p2→m2 ·M mb→p3 -1 ·(x p2针 ,y p2针 ,1) T (Δx2,Δy2)=(x 2针 -x m2c +x m2针 ,y 2针 -y m2c +y m2针 )。 10. A multi-camera probe device used in the probe calibration method according to claim 9, characterized in that: include: A base frame, a first driving module, a second driving module, a first test axis, a second test axis and a third camera; The first driving module and the second driving module are arranged on the base frame, the first test shaft is installed on the first driving module, and the second test shaft is installed on the second driving module; The first test axis includes a first bracket, a first camera and a first probe, the first bracket is mounted on the first driving module, and the first camera and the first probe are mounted on the first bracket; The second test axis includes a second bracket, a second camera, and a second probe. The second bracket is mounted on the second driving module, and the second camera and the second probe are mounted on the second bracket.
11. The multi-camera probe device according to claim 10, characterized in that: The base frame is also provided with a calibration plate fixing part, and the third camera is fixed below the calibration plate fixing part.
12. A flying probe tester, characterized in that: It comprises a memory, a processor and a multi-camera probe device as claimed in claim 10 or 11, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the probe calibration method as claimed in claim 9.
