Microscopic visual detection device and calibration method for cross-scale micro-nano device assembly

Abstract

This invention provides a microscopic visual inspection device and calibration method for multi-scale micro / nano device assembly, including a first microscopic vision system and a second microscopic vision system for detecting front and side features of micro / nano devices; a third microscopic vision system for detecting top features of micro / nano parts; and a fourth microscopic vision system for detecting bottom features of micro / nano devices. It also includes a calibration method for the microscopic visual inspection device for multi-scale micro / nano device assembly. Cross-depth-of-field detection is achieved by calibrating the image offset matrix of the microscopic vision system and the image Jacobian matrix for part position and motion control. A relational matrix is ​​derived, and combined with the relational matrix calibration method, the detection of part angular deviation around the Z-axis is achieved. This invention is applicable to microscopic visual inspection of multi-scale micro / nano device assembly, enabling multi-angle detection, and simultaneously detecting part rotational angular deviation through relational derivation.

Description

Technical Field

[0001] This invention relates to the field of micro-part assembly technology, and more specifically, to a microscopic visual inspection device and calibration method for multi-scale micro / nano device assembly. Background Technology

[0002] Currently, most microscopic vision systems in micro-assembly systems consist of multi-channel microscopic vision. These multi-channel microscopic vision systems are arranged in a specific spatial layout to observe the relative state of parts during the assembly process from different angles. The microscopic vision detects the pose deviation of the part in the image space. By calibrating the multi-channel microscopic vision relationship matrix and combining it with the calibration of the image space to Cartesian space relationship matrix, the spatial pose detection of the part based on the fusion of multi-channel microscopic vision information is realized, thereby achieving part assembly control.

[0003] The Institute of Automation, Chinese Academy of Sciences, proposed a method for spatial pose detection and alignment of micro-parts based on three-channel orthogonal microscopic vision (Zhang Juan, Xu De, Zhang Zhengtao, et al. An automatic alignment device and method for micro-parts based on multi-channel microscopic vision, patent number CN103273310B). This method achieves spatial pose detection and alignment of two parts by calibrating the image Jacobian matrix of the three-channel microscopic vision servo motion control. Harbin University of Science and Technology (Qu Jiwang, Xu Jiazhong, Zhang Dapeng, et al. Precision assembly of microspheres and tubes based on microscopic vision [J]. High Technology Communications, 2019, 9: 914-924) constructed a four-channel microscopic vision detection system for the precision assembly of microspheres and microtubes. Two low-magnification microscopic vision channels are used for coarse positioning at horizontal orthogonal positions, while two high-magnification microscopic vision channels are at specific angles to the horizontal plane and are orthogonally arranged to achieve precise positioning of the microspheres and microtubes. By calibrating the relationship matrix from the horizontal two-way microscopic visual image space to the Cartesian space, microtube attitude adjustment and coarse positioning were achieved; by calibrating the relationship matrix from the oblique two-way microscopic visual image space to the Cartesian space, fine alignment of the microtube position was achieved.

[0004] Current microscopic vision inspection systems consist of multiple microscopic vision channels arranged at specific spatial angles. These systems occupy a large space, making them unsuitable for micro-assembly systems including 6DOF robotic arms. Furthermore, current systems can only inspect the top and sides of parts, failing to detect bottom features, thus limiting their application scope. While existing calibration techniques convert image spatial pose deviations from different orientations to Cartesian spatial pose deviations, they do not address the cross-depth-of-field detection problem in multi-scale micro / nano device assembly. This lack of consideration for image offset matrix calibration during microscopic vision focusing results in low cross-depth-of-field detection accuracy. Moreover, current calibration methods cannot calibrate the top and bottom vision relationship matrix of multi-scale micro / nano device assembly systems, thus failing to detect Z-axis angular deviations of parts. Summary of the Invention

[0005] The purpose of this invention is to provide a microscopic visual inspection device and calibration method for multi-scale micro / nano device assembly, which solves the problems of multi-angle detection of micro parts and detection of Z-axis angle deviation of parts in multi-scale micro / nano device assembly system.

[0006] The embodiments of the present invention are achieved through the following technical solution: a microscopic visual inspection device for assembling micro-nano devices across scales, comprising an operating table connected to a horizontal plane for assembling micro-nano devices and a robotic arm for moving micro-nano devices. The robotic arm is connected to a robotic arm end adapter plate, and the robotic arm end adapter plate is connected to a part A holder, which holds part A. The operating table is connected to a part B holder, which holds part B. The device includes a first microscopic visual system and a second microscopic visual system. The first microscopic visual system, the second microscopic visual system, and the robotic arm are all arranged with the operating table as the center. The optical axes of the first microscopic visual system and the second microscopic visual system are both parallel to the horizontal plane, and they present an approximately 90-degree angle in spatial arrangement. The device is used for detecting the front and side features of micro-nano devices. The robotic arm is positioned directly opposite the second microscopic visual system.

[0007] A third microscopic vision system is connected to the end-effector plate of the robotic arm for detecting top features of micro / nano parts.

[0008] A fourth microscopic vision system, wherein the optical axis of the fourth microscopic vision system is perpendicular to the horizontal plane, is used for bottom feature detection of micro- and nano-devices.

[0009] A calibration method for a microscopic visual inspection device assembled using multi-scale micro / nano devices, characterized in that the calibration method includes:

[0010] Step S1: Establish the coordinate system of the multi-scale micro / nano device assembly system, including the first microscopic vision system coordinate system P. w1 The second microscopic vision system coordinate system P w2 The third microscopic vision system coordinate system P w3 The fourth microscopic visual coordinate system P w4 The coordinate system P of the control panel w5 The end effector coordinate system P of the robotic arm w6 Robotic arm base coordinate system P w Cartesian coordinate system P wo Furthermore, the establishment of all coordinate systems conforms to the right-hand rule; through the calibration relationship, the spatial changes in the image are converted into the motion changes in the Cartesian space, thereby achieving detection;

[0011] Step S2: The image offset matrix J of the first microscopic vision system is used to achieve the active motion of the first microscopic vision system and the second microscopic vision system respectively. B1 Image offset matrix J of the second microscopic vision system B2 Calibration;

[0012] Step S3: Calibrate the image Jacobian matrix of the position and motion control of the first and second microscopic vision systems to determine the coordinates of part A and part B in the robot arm's base coordinate system P. w The X, Y, and Z axis position deviations d X d Y d Z Calculation;

[0013] Step S4; Calibrate the image Jacobian matrix of the first and second microscopic vision systems for angular motion control, which is used to calculate the angular deviations of part A and part B around the X-axis and Y-axis.

[0014] Step S5: Calibrate the relationship matrix between the third and fourth microscopic vision systems, used to determine the rotational angle deviation between part A and part B. calculate.

[0015] Furthermore, J is calculated according to the following formula. B1 and J B2：

[0016] (1)

[0017] J is calculated using the linear least squares method. B1 As shown in formula (2), the image offset matrix J of the second microscopic vision system B2 Adopted with J B1 The same calibration method can be used to obtain

[0018] (2)

[0019] Where n is the number of times the first microscopic vision system moves along the focusing axis; (i=1,2,...n) represents the motion changes of the first microscopic visual system; , ) represents the change in the image coordinates of the calibration object.

[0020] Furthermore, d is calculated according to the following formula. X d Y d Z：

[0021] (3)

[0022] Where X is the robot arm along the base coordinate system P w Let C be the matrix of motion changes formed by n movements, and C be the matrix of image position changes of the calibration object formed by n movements. The image Jacobian matrix J can be obtained using the least squares method. V As shown in formula (4)

[0023] (4)

[0024] Through J V The spatial positional deviation (d) between part A and part B is calculated. X d Y d Z As shown in formula (5),

[0025] (5)

[0026] in and These represent the image offset matrices for the first and second microscopic visions calibrated in step 1, respectively. The first microscopic vision system focuses on the motion changes of parts A and B along the focusing axis, respectively. The second microscopic vision system focuses the motion changes of parts A and B along the focusing axis, respectively. , )and( , The figures represent the image position deviations of parts A and B in the first and second microscopic vision systems, respectively.

[0027] A calibration method for a microscopic visual inspection device assembled using multi-scale micro / nano devices is proposed, which calculates the following formula. , :

[0028] (6)

[0029] Where χ represents the calibration object along the robot arm's base coordinate system P w1 The angle change matrix is ​​given by θ, which is the matrix representing the nth degree of angle change of the calibration object in the image space. Using the least squares method, the image Jacobian matrix J in formula (6) can be obtained. R As shown in formula (7)

[0030] (7)

[0031] Through J R The spatial angular deviation between part A and part B was calculated. , As shown in formula (8),

[0032] (8)

[0033] in , The image angle deviations of part A and part B in the first and second microscopic vision systems are respectively.

[0034] Furthermore, calculate according to the following formula. :

[0035] (9)

[0036] Where R p P is the coordinate system of the third microscopic vision system. w3 coordinate system P of the end effector of the robotic arm w6 The rotation relation matrix; , , Let P be the coordinate system of part A and part B in the third microscopic vision system. w3 Positional deviation, ( , , )for( , , Transform to the end-effector coordinate system Pw6 Positional deviation;

[0037] (10)

[0038] Where N is the coordinate system of the control robot arm at the end of the robot arm tool P w6 The number of times the motion occurs in the XY plane (n≥3), , ,0) represents the changes in motion of the robotic arm (i=1,2,...n) and ( , ) represents the change in image coordinates of the calibration object in the third microscopic vision system, k w3 The image scaling factor for the third microscopic vision system;

[0039] (11)

[0040] Where R p This was obtained using the linear least squares method;

[0041] (12)

[0042] in( , , Let P be the coordinate system of part A and part B at the end of the robotic arm. w6 Positional deviation, ( , , )for( , , Transformation to base coordinate system P w Positional deviation, , , It can be read directly from the robotic arm's teach pendant without the need for recalibration;

[0043] (13)

[0044] in( , , Let P be the coordinate system of part A and part B in the fourth microscopic vision system. w4 Positional deviation;

[0045] (14)

[0046] Where n is the coordinate system of the robotic arm in the base coordinate system P. w The number of times the motion occurs in the XY plane (n≥3), , (i=1,2,...n) represents the change in motion of the robotic arm; , () represents the change in the image coordinates of the calibration object in the fourth microscopic vision system;

[0047] (15)

[0048] Where R o Calculated using the linear least squares method;

[0049] (16)

[0050] Where R m P is the coordinate system of the third microscopic vision system. w3 With the fourth microscopic vision system coordinate system P w4 The rotation matrix;

[0051] (17)

[0052] in Let part B be in the coordinate system P of the third microscopic vision system. w3 attitude around the Z-axis The transformed coordinates in the fourth microscopic vision system P w4 The orientation around the Z-axis;

[0053] (18)

[0054] in Let part A be in the fourth microscopic vision system coordinate system P w4 Attitude detection around the Z-axis.

[0055] The technical solution of the present invention has at least the following advantages and beneficial effects: by using the structure of "three fixed microscopic vision + one follow-up microscopic vision", the relative pose state of the parts during the assembly process can be detected from four different angles. One of the follow-up microscopic vision paths is installed at the end of the 6DOF tandem robotic arm, enabling the detection of features on the top of the part. Two of the three fixed microscopic vision paths are orthogonally arranged on the horizontal plane, enabling the detection of features on the sides and front of the part. The optical axis of the third microscopic vision path is perpendicular to the horizontal plane, enabling the detection of features from the bottom of the part. Simultaneously, a calibration method was used to convert the image pose detection deviation of the part into a Cartesian spatial pose deviation. First, by calibrating the image offset matrix of the microscopic vision system and the image Jacobian matrix for part position motion control, the detection of part X, Y, and Z axis position deviations under cross-depth-of-field detection conditions was achieved. Second, by calibrating the image Jacobian matrix for angle motion control, the detection of part angular deviations around the X and Y axes was achieved. Then, by deriving the relationship matrix between the top and bottom vision paths and combining it with the calibration of key parameters in the relationship matrix, the detection of part angular deviations around the Z axis was achieved. Finally, the detection of 6DOF spatial pose of parts during the assembly of various types of cross-scale micro-nano devices was realized, improving the detection accuracy. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A schematic diagram of the structure of the cross-scale micro-nano device assembly system of the microscopic vision inspection device for cross-scale micro-nano device assembly provided in an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of the coordinate system of the cross-scale micro-nano device assembly system for the calibration method of the microscopic vision inspection device assembled using cross-scale micro-nano devices provided in the embodiments of the present invention;

[0059] Figure 3 The images shown are of small ball images from a microscopic vision inspection device for multi-scale micro / nano device assembly provided in this embodiment of the invention; wherein (a) is the image detection result of the first microscopic vision system; and (b) is a schematic diagram of the image detection result of the second microscopic vision system.

[0060] Figure 4 The image detection results of the metal rod of the microscopic vision inspection device for multi-scale micro-nano device assembly provided in the embodiments of the present invention; wherein (a) is the image detection result of the metal rod of the first microscopic vision system; and (b) is a schematic diagram of the image detection result of the metal rod of the second microscopic vision system.

[0061] Figure 5 The images shown are: (a) the image detection results of the ball after the first movement of the ball in the third microscopic vision system of the microscopic vision inspection device for multi-scale micro / nano device assembly provided in the embodiments of the present invention; (b) the image detection results of the ball after the second movement of the ball in the third microscopic vision system; (c) the image detection results of the ball after the third movement of the ball in the third microscopic vision system; and (d) a schematic diagram of the image detection results of the ball after the fourth movement of the ball in the third microscopic vision system.

[0062] Figure 6The images shown are: (a) the image detection results of the ball after the first movement of the ball in the fourth microscopic vision system of the microscopic vision inspection device for multi-scale micro / nano device assembly provided in the embodiments of the present invention; (b) the image detection results of the ball after the second movement of the ball in the fourth microscopic vision system; (c) the image detection results of the ball after the third movement of the ball in the fourth microscopic vision system; and (d) a schematic diagram of the image detection results of the ball after the fourth movement of the ball in the fourth microscopic vision system.

[0063] Icons: 1-First microscopic vision system, 2-Second microscopic vision system, 3-Third microscopic vision system, 4-Fourth microscopic vision system, 5-Operating table, 6-Robotic arm, 7-Part A, 8-Part B, 9-Robotic arm end effector plate, 10-Part A gripper, 11-Part B gripper, 21-Coordinate system P of the first microscopic vision system w1 ,22-Second Microscopic Vision System Coordinate System P w2 ,23-Third Microscopic Vision System Coordinate System P w3 ,24-Fourth Microscopic Visual Coordinate System P w4 ,25-Operating console coordinate system P w5 ,26- Robotic arm end-effector coordinate system P w6 ,27-Robot arm base coordinate system P w ,28-Cartesian coordinate system P wo . Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0065] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0066] Example

[0067] This embodiment provides a microscopic visual inspection device for assembling micro / nano devices across scales, such as... Figure 1As shown, the system includes an operating platform 5 for assembling micro / nano devices connected to a horizontal plane and a robotic arm 6 for moving the micro / nano devices. The robotic arm is connected to a robotic arm end-effector 9, which is connected to a part A holder 10. The part A holder holds part A7. The operating platform 5 is connected to a part B holder 11, which holds part B8. The system includes a first microscopic vision system 1 and a second microscopic vision system 2. The first microscopic vision system 1, the second microscopic vision system 2, and the robotic arm 6 are all centered on the operating platform 5. The optical axes of the first microscopic vision system 1 and the second microscopic vision system 2 are both parallel to the horizontal plane, and they are arranged at approximately a 90-degree angle in spatial layout; this is used for front and side feature detection of micro / nano devices; the robotic arm 6 is positioned directly opposite the second microscopic vision system 2; the third microscopic vision system 3 is connected to the end adapter plate 9 of the robotic arm; this is used for top feature detection of micro / nano parts; the fourth microscopic vision system 4 has its optical axis perpendicular to the horizontal plane, and is used for bottom feature detection of micro / nano devices;

[0068] It is worth mentioning that one of the follow-up microscopic vision channels is installed at the end of the 6DOF tandem robotic arm, realizing the detection of features on the top of the part; two of the three fixed microscopic vision channels are arranged orthogonally to each other on the horizontal plane, realizing the detection of features on the side and front of the part; the optical axis of the other microscopic vision channel is perpendicular to the horizontal plane, realizing the detection of features from the bottom of the part; through the structure of "three fixed microscopic vision channels + one follow-up microscopic vision channel", the relative pose of the part during the assembly process can be detected from four different angles.

[0069] Establishing the coordinate system for a multi-scale micro / nano device assembly system, such as Figure 2 As shown, it includes the first microscopic vision system coordinate system P. w1 21; Second microscopic vision system coordinate system P w2 22; Third microscopic vision system coordinate system P w3 23; Fourth Microscopic Visual Coordinate System P w4 24; Control panel coordinate system P w5 25; Robotic arm end-effector coordinate system P w6 26; Robotic arm base coordinate system P w 27; Cartesian coordinate system P wo 28. All coordinate systems are established according to the right-hand rule. Cartesian coordinate system P wo The XY plane of the 28-axis remains parallel to the horizontal plane. The robot arm's base coordinate system P... w 27. Taking the center of the robotic arm chassis as the origin, its coordinate system P is parallel to the Cartesian coordinate system P. wo 28 remains parallel in both the X and Y directions. The end effector coordinate system P of the robotic arm.w6 26. With the center of the robotic arm's end effector as the origin, the coordinate system P of the robotic arm's end effector is... w6 26 and the robot arm's base coordinate system P w The relationship 27 is determined by the robotic arm manufacturer at the factory and can be read in real time via the robotic arm teach pendant. As the robotic arm's pose changes, the coordinate system P at the robotic arm's end effector... w6 26 and the coordinate system P of the robotic arm base w 27 presents a specific angle. The control panel coordinate system P w5 25. Taking the center of the bottom of the control panel as the origin, its coordinate system P is parallel to the Cartesian coordinate system P. wo 28. The first and second microscopic vision systems (system 1 and system 2) are parallel and aligned in the X and Y directions. Both systems are arranged on a horizontal plane and are orthogonal to each other. The coordinate system of the first microscopic vision system is P. w1 21. Taking the intersection of the optical axis of the microscope lens and the CCD target surface as the origin of the coordinate system, its coordinates are related to the Cartesian coordinate system P. wo 28 are parallel and aligned in the X and Y directions. The second microscopic vision system coordinate system P w2 22. Taking the intersection of the optical axis of the microscope lens and the CCD target surface as the origin of the coordinate system, its coordinates are related to the Cartesian coordinate system P. wo 28 are parallel and aligned in the X and Y directions. The third microscopic vision system 3 is mounted at the end of the robotic arm 6, and the coordinate system of the third microscopic vision system is P. w3 23. Taking the intersection of the optical axis of the microscope lens and the CCD target surface as the origin of the coordinate system, and the coordinate system P at the end of the robotic arm... w6 26. Keep parallel in the Z-axis direction. The fourth microscopic vision system 4 is mounted on a horizontal plane. Fourth microscopic vision coordinate system P w4 24. Taking the intersection of the optical axis of the microscope lens and the CCD target surface as the origin of the coordinate system, and the coordinate system P at the end of the robotic arm... w6 26. Maintain parallelism along the Z-axis. Through calibration, image spatial variations are converted into Cartesian spatial motion variations for detection.

[0070] By calibrating the image Jacobian matrix of the first microscopic vision system 1 and the second microscopic vision system 2 for position and motion control, the positions of parts A7 and B8 in the robot arm's base coordinate system P are realized. w X, Y, Z axis position deviations d under 27 X d Y d Z The calculation; by calibrating the image Jacobian matrix of the first microscopic vision system 1 and the second microscopic vision system 2 to control the angle motion, the angular deviations of part A7 and part B8 around the X-axis and Y-axis are realized. , The calculation; the rotation angle deviation between part A7 and part B8 is realized through the third microscopic vision system 3 and the fourth microscopic vision system 4. The calculation of this multi-scale micro / nano device assembly system calibration method includes the following steps:

[0071] Step S1: The image offset matrix J of the first microscopic vision system 1 is realized through the active motion of the first microscopic vision system 1 and the second microscopic vision system 2 respectively. B1 Image offset matrix J of the second microscopic vision system B2 The calibration is performed. The image offset matrix describes the image point offset caused by the linear motion of the microscopic vision system along the focusing axis. The calibration object is fixed in the field of view of the first microscopic vision system 1. The first microscopic vision system 1 moves n times (n≥3) along the focusing axis, obtaining n+1 known positions. When the motion change of the first microscopic vision system 1 is (i=1,2,...n), the change in the image coordinates of the calibration object is obtained through image processing techniques and denoted as ( , If ), then the relationship shown in formula (1) exists.

[0072] (1)

[0073] J is calculated using the linear least squares method. B1 As shown in formula (2). The image offset matrix J of the second microscopic vision system 2 B2 Adopted with J B1 The same calibration method can be used to obtain the result.

[0074] (2)

[0075] Step 2: Calibrate the image Jacobian matrix of the first microscopic vision system 1 and the second microscopic vision system 2 to realize the position and motion control of parts A7 and B8 in the robot arm base coordinate system P. w X, Y, Z axis position deviations d under 27 X d Y d Z The calculation involves first calibrating the image Jacobian matrix J, which represents the change in translational motion of the robotic arm 6 to the change in translational motion in the image space. V Install the calibration object at the end of the robotic arm 6, and make the robotic arm 6 move along the robotic arm base coordinate system P. w 27. Perform n movements (n≥3), and record the motion of the robotic arm 6. x i , y i , z i(i=1,2,...n), after each movement of the robotic arm 6, the first microscopic vision system 1 and the second microscopic vision system 2 change their motion along the focusing axis to achieve focusing of the calibration object. and And the change in coordinates of the calibrated object in the first microscopic vision system 1 ( , ) and the coordinate change in the second microscopic vision system 2 ( , The relationship between the change in image spatial motion and the change in Cartesian spatial motion is shown in formula (3).

[0076] (3)

[0077] Where X is the coordinate system of the robotic arm 6 along the base coordinate system P w 27 is the matrix of motion changes formed by n movements, and C is the matrix of image position changes of the calibration object formed by n movements. and These represent the image offset matrices for the first and second microscopic visions, respectively, as determined in step 1. The image Jacobian matrix J can be obtained using the least squares method. V As shown in formula (4).

[0078] (4)

[0079] Then, through J V The spatial positional deviation (d) between part A7 and part B8 was calculated. X d Y d Z As shown in formula (5), where The first microscopic vision system 1 focuses the motion changes of parts A7 and B8 along the focusing axis, respectively. The second microscopic vision system 2 focuses the motion changes of parts A7 and B8 along the focusing axis, respectively. , )and( , The image position deviations of parts A7 and B8 in the first microscopic vision system 1 and the second microscopic vision system 2 are respectively.

[0080] (5)

[0081] Step S3: Calibrate the image Jacobian matrix for the angle motion control of the first microscopic vision system 1 and the second microscopic vision system 2 to realize the angular deviation of part A and part B around the X-axis and Y-axis. , The calculation involves first calibrating the image Jacobian matrix J, which represents the change in angle of the robotic arm from 6 angles to the change in angle in the image space. R The robotic arm 6 moves the calibration object around the robotic arm's base coordinate system P. w1 The robot arm moves along the X and Y axes n times (n≥3), and the angular changes of the robot arm around the X and Y axes of the base coordinate system are recorded respectively. α i and β i , (i=1,2,...n), and the angular change of the calibration object in the first microscopic vision system 1. The change in angle with the second microscopic vision system 2 Then, robotic arm 6 moves along the base coordinate system P. w1 The relationship between the spatial motion change of 21 and the spatial motion change of the image is shown in formula (6).

[0082] (6)

[0083] Where χ represents the calibration object along the robot arm base coordinate system P w1 The angular change matrix is ​​21, where θ is the nth degree angular change matrix of the calibration object in the image space. Using the least squares method, the image Jacobian matrix J in formula (6) can be obtained. R As shown in formula (7).

[0084] (7)

[0085] Then through J R The spatial angular deviation between part A and part B was calculated. , As shown in formula (8), where , The image angle deviations of parts A7 and B8 in the first microscopic vision system 1 and the second microscopic vision system 2 are respectively.

[0086] (8)

[0087] Step S4: Calibrate the relationship matrix between the third microscopic vision system 3 and the fourth microscopic vision system 4 to realize the rotation angle deviation between part A7 and part B8. The calculation.

[0088] Step S4-1: Derive the coordinate system P of the third microscopic vision system w3 23 and the coordinate system P of the end effector of the robotic arm w6 The rotation relation matrix R of 26 pIf mechanical installation angle errors are not considered, the coordinate system P of the third microscopic vision system w3 23. Rotating clockwise by α around the Z-axis will align the tool coordinate system P of the end effector. w6 If 26 are parallel, then parts A7 and B8 lie in the third microscopic vision system coordinate system P. w3 23 positional deviation ( , , Transform to the end-effector coordinate system P w6 26 positional deviation ( , , As shown in formula (9).

[0089] (9)

[0090] The calibration object is clearly presented within the focal plane of the third microscopic vision system 3, and then the robotic arm is controlled to move in the tool coordinate system P at the end of the robotic arm. w6 The robot arm moves n times in the XY plane (n≥3), and the changes in motion are recorded respectively. , The changes in the image coordinates of the calibrator in the third microscopic vision system 3 are (i=1,2,...n) and (i=0,2,...n). , Then the relationship shown in formula (10) exists, where k w3 is the image scaling factor for the third microscopic vision system 3.

[0091] (10)

[0092] R is calculated using the linear least squares method. p As shown in formula (11).

[0093] (11)

[0094] Step S4-2: Derive the coordinate system P of the robotic arm end effector. w6 26 and the robot arm's base coordinate system P w The rotation relationship matrix of 27. Part A7 and part B8 in the end-effector tool coordinate system P. w6 26 positional deviation ( , , Transformation to base coordinate system P w 27 positional deviation ( , , As shown in formula (12). In the current state, the coordinate system P of the robotic arm's end effector is... w6 26 First around Pw6 26 X-axis clockwise rotation angle Then around P w6 26 Y-axis clockwise rotation angle Finally, around P w6 26 Z-axis clockwise rotation angle Afterwards, the coordinate system P of the robotic arm is reconnected. w 27 parallel.

[0095] (12)

[0096] Due to the coordinate system P of the end effector of the robotic arm w6 26 and the robot arm's base coordinate system P w The relationship between 27 and 27 is set at the factory when the robotic arm leaves the factory. , , The readings can be directly obtained from the robotic arm's teach pendant without the need for recalibration.

[0097] Step S4-3: Derive the coordinate system P of the fourth microscopic vision system w4 24 and the robot arm base coordinate system P w The rotation relation matrix R of 27 o Since both the fourth microscopic vision system and the robotic arm are mounted on the base platform, if mechanical mounting angle errors are disregarded, the coordinate system P of the fourth microscopic vision system... w4 24. Rotate clockwise around the Z-axis relative to the robot arm's base coordinate system P w 27. Parallel. Part A7 and Part B8 are in the fourth microscopic vision system coordinate system P. w4 24 positional deviation ( , , Transform to the robot arm's base coordinate system P w The positional deviation of 27 is shown in formula (13).

[0098] (13)

[0099] The calibration object is clearly presented within the focal plane of the fourth microscopic vision system 4, and then the robotic arm is controlled to move in the robotic arm base coordinate system P. w The robot arm moves n times in the XY plane (n≥3), and the changes in motion are recorded respectively. , The changes in the image coordinates of the calibration object in the fourth microscopic vision system 4 are (i=1,2,...n) and (i=0,2,...n). , Then the relationship shown in formula (14) exists, where k w4 , which is the image scaling factor of the fourth microscopic vision system 4.

[0100] (14)

[0101] R is calculated using the linear least squares method. o As shown in formula (15).

[0102] (15)

[0103] By deriving formulas (9), (12), and (13), the coordinate system P of the third microscopic vision system can be obtained. w3 23 and the fourth microscopic vision system coordinate system P w4 The rotation matrix R of 24 m Parts A7 and B8 are located in the third microscopic vision system coordinate system P. w3 The positional deviation of 23 is transformed into the fourth microscopic vision system coordinate system P. w4 The positional deviation of 24 is shown in formula (16).

[0104] (16)

[0105] Step S4-4: Realize the coordinate system P of the robotic arm end-effector tool through motion control. w6 26 and the robot arm's base coordinate system P w 27 in P w If the XY planes of 27 are parallel, and mechanical installation errors are ignored, then the part A7 at the end of the robot arm is parallel to P. w The XY planes of 27 are parallel. Since the control panel is also mounted on the base platform, if mechanical installation angle errors are disregarded, part B8 on the control panel is parallel to P. w The XY planes of part B are parallel. Therefore, part B lies in the third microscopic vision system coordinate system P. w3 The orientation of 23 around the Z-axis is converted into the coordinate system P of the fourth microscopic vision system. w4 The orientation of 24 around the Z-axis is shown in formula (17).

[0106] (17)

[0107] Part A7 in the fourth microscopic vision system coordinate system P w4 The attitude detection of 24 around the Z-axis is as follows: the rotation attitude calculation formula of part A7 and part B8 is as shown in formula (14).

[0108] (18) Detailed implementation method:

[0110] Step S1: The image offset matrix J of the first microscopic vision system 1 is realized through the active motion of the first microscopic vision system 1 and the second microscopic vision system 2 respectively.B1 Image offset matrix J of the second microscopic vision system 2 B2 The calibration.

[0111] 1) Calibration is performed using a step calibration block, which is stepped and has five steps. The step calibration block is placed in the image field of view of the first microscopic vision system 1, so that the first step is clearly imaged, and the current image position coordinates of the first step are recorded. The image position coordinates (u0, v0) of the center point of the step calibration block are calculated by using the geometric prior knowledge of the step calibration object and the image scale coefficient.

[0112] 2) The first microscopic vision system 1 moves along the camera focusing axis, making the second step clearly visible in the field of view of the first microscopic vision system. The motion amount l1 of the first microscopic vision system 1 is recorded, as well as the image position coordinates of the second step after the motion. The image position coordinates (u1, v1) of the center point of the step calibration block are calculated using the geometric prior knowledge of the step calibration object and the image scaling factor. The image position deviation (Δ) of the center point of the step calibration object is then obtained. u 1, Δ v 1).

[0113] 3) Repeat the process in step 2), focusing on the third, fourth, and fifth steps in sequence to obtain the motion of the four sets of the first microscopic visual system 1. l i (i=1,2,3,4) and the difference (Δ) between the image position coordinates of the center point of the step calibration object before and after its movement u i , Δ v i ).

[0114] 4) Substitute the data into formula (2) to obtain the image offset matrix J B1 .

[0115] The image offset matrix J of the second microscopic vision system 2 is obtained by calibration using the above process. B2 .

[0116] Step S2: Calibrate the Jacobian matrix of the position and motion control images of the first microscopic vision system 1 and the second microscopic vision system 2 to realize the X, Y, and Z axis position deviations d between the sleeve part and the cavity part. X d Y d Z The calculation.

[0117] 1) Place the ball within the image fields of the first microscopic vision system 1 and the second microscopic vision system 2. By adjusting the spatial positions of the first microscopic vision system 1 and the second microscopic vision system 2, achieve a clear image of the ball. Record the coordinates (u) of the center position of the ball in the first microscopic vision system 1. 1o,v 1o ) and the center image position coordinates (u) of the second microscopic vision system 2 2o ,v 2o ),like Figure 3 As shown.

[0118] 2) The robotic arm moves the ball in space, with random motion values. The movement values ​​of the robotic arm (Δx1, Δy1, Δz1) are recorded. After the movement, the first microscopic vision system 1 and the second microscopic vision system 2 focus on the ball respectively. First, the changes in motion of the first microscopic vision system 1 and the second microscopic vision system 2 are recorded. and Then, the coordinates of the center position of the ball in the images of the first microscopic vision system 1 and the second microscopic vision system 2 are recorded (u 11 ,v 11 ) and (u 21 ,v 21 ), to obtain the image position deviation (Δu) 11 , Δv 11 ) and (Δu 21 , Δv 21 ).

[0119] 3) Repeat step 2) five times to obtain five sets of robotic arm motion values ​​(Δx). i , Δy i , Δz i (i=1,2,...,5), the motion changes of the first microscopic vision system 1 and the second microscopic vision system 2. and And the image position deviation of the center of the ball before and after the motion (Δu) 1i , Δv 1i ) and (Δu 2i , Δv 2i ).

[0120] 4) Substitute the data into the above formula (4) to obtain the image Jacobian matrix J. V .

[0121] 5) The image positional deviations between the sleeve and the gold cavity in the first microscopic vision system 1 and the second microscopic vision system 2 were calculated using image feature extraction algorithms. , )and( , Substitute the data into formula (5) to calculate d. X d Y d Z .

[0122] Step S3: Calibrate the Jacobian matrix of the angle motion control images of the first microscopic vision system 1 and the second microscopic vision system 2 to realize the angular deviation of the sleeve and the gold cavity around the X-axis and around the Y-axis. , The calculation.

[0123] 1) Control the pose adjustment of the metal rod to ensure that it is clearly imaged in both the first microscopic vision system 1 and the second microscopic vision system 2, and record the image feature angle of the metal rod in the first microscopic vision system 1 as α. 1o The image feature angle of the second microscopic vision system 2 is β. 1o ,like Figure 4 As shown.

[0124] 2) The robotic arm drives the metal rod to rotate around the robotic arm's base coordinate system P. w The X and Y axes of 27 are rotated respectively, with random values ​​for the rotation angles. The rotation angle values ​​are recorded as (△). α1 , △ β1 ), and record the angular deviation of the image features of the metal rod between the first microscopic vision system 1 and the second microscopic vision system 2 as (△θ). x1 , △θ y1 ).

[0125] 3) Repeat step 2) five times to obtain five sets of motion values ​​(Δ) for the rotating platform. αi ,△ βi (i=1,2,...5) and the corresponding image feature angle deviation (△θ) before and after the metal rod moves. xi , △θ yi ).

[0126] 4) Substitute the data into the above formula (7) to obtain the image rotation calibration matrix J. R .

[0127] Step S4: Calibrate the relationship matrix R between the third microscopic vision system 3 and the fourth microscopic vision system 4. m Achieving the rotational angle deviation between the sleeve part and the cavity part The calculation.

[0128] 1) Make the ball clearly visible in the focal plane of the third microscopic vision system 3, and then control the robotic arm in the tool coordinate system P at the end of the robotic arm. w6 The robotic arm moves 5 times in the XY plane, and the changes in motion are recorded for each movement. , The changes in the image coordinates of the calibrator in the third microscopic vision system 3 are (i=1,2,...5) and (i=0,2,...5). , ),like Figure 5As shown. Substituting the data into formula (11) yields R. p .

[0129] 2) Read out via robotic arm teach pendant , , Then, R is calculated using formula (12). v .

[0130] 3) Ensure the calibration object is clearly presented within the focal plane of the fourth microscopic vision system 4, and then control the robotic arm in the robotic arm base coordinate system P. w The robotic arm moves 5 times in the XY plane, and the changes in motion are recorded for each movement. , The changes in the image coordinates of the calibrator in the fourth microscopic vision system 4 are (i=1,2,...5) and (i=0). , ),like Figure 6 As shown. Ro is then calculated using formula (15).

[0131] 4) Calculate R using formula (16) m .

[0132] 5) Control the robot arm's posture adjustment so that the robot arm's end-effector coordinate system P w6 26 and the robot arm's base coordinate system P w1 21. If the robot arm's end sleeve remains parallel to the horizontal plane, then the orientation of the sleeve at the end of the robot arm will also remain parallel to the horizontal plane. The orientation of the metal cavity on the operating table will also remain parallel to the horizontal plane. Therefore, the metal cavity will be in the third microscopic vision system coordinate system P. w3 23's orientation around the Z-axis The equation (17) is converted to the coordinate system P of the fourth microscopic vision system. w4 24's orientation around the Z-axis The sleeve in the fourth microscopic vision system coordinate system P w4 The attitude detection of 24 around the Z-axis is The rotational attitude deviation between the sleeve and the cavity is calculated by substituting it into formula (18).

[0133] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A calibration method for a microscopic visual inspection device using multi-scale micro / nano device assembly, the multi-scale micro / nano device assembly microscopic visual inspection device comprising an operating table (5) connected to a horizontal plane for assembling micro / nano devices and a robotic arm (6) for moving micro / nano devices, the robotic arm (6) being connected to a robotic arm end adapter plate (9), the robotic arm end adapter plate (9) being connected to a part A holder (10), the part A holder (10) holding part A (7), the operating table (5) being connected to a part B holder (11), the part B holder (11) holding part B (8), comprising: The first microscopic vision system (1) and the second microscopic vision system (2) are arranged around the operating table (5). The optical axes of the first microscopic vision system (1) and the second microscopic vision system (2) are parallel to the horizontal plane and are arranged at an angle of approximately 90 degrees in space. The first microscopic vision system (1) and the second microscopic vision system (2) are arranged around the horizontal plane ... The third microscopic vision system (3) is connected to the end adapter plate (9) of the robotic arm for detecting the top features of micro-nano devices. The fourth microscopic vision system (4) has its optical axis perpendicular to the horizontal plane and is used for bottom feature detection of micro-nano devices; The calibration method is characterized by comprising: Step S1: Establish a coordinate system, including the first microscopic vision system coordinate system P. w1 (21); Second microscopic vision system coordinate system P w2 (22); Third microscopic vision system coordinate system P w3 (23); Fourth microscopic vision system coordinate system P w4 (24); Control panel coordinate system P w5 (25); Robotic arm end coordinate system P w6 (26); Robotic arm base coordinate system P w (27); Cartesian coordinate system P wo (28), and the establishment of all coordinate systems conforms to the right-hand rule; among them, the Cartesian space coordinate system P wo (28) The XY plane remains parallel to the horizontal plane, and the base coordinate system P of the robotic arm is... w (27) Taking the center of the robotic arm chassis as the origin, its coordinate system P is parallel to the Cartesian coordinate system P. wo (28) The coordinate system of the end effector of the robotic arm remains parallel in both the X and Y directions. w6 (26) With the center of the end effector of the robotic arm as the origin, the coordinate system P of the operating table is... w5 (25) Taking the center of the bottom of the operating table as the origin, its coordinate system P is perpendicular to the Cartesian coordinate system P. wo (28) The first microscopic vision system (1) and the second microscopic vision system (2) are arranged on a horizontal plane and orthogonally to each other, keeping them parallel and in the same direction in the XY direction. The coordinate system P of the first microscopic vision system is... w1 (21) Taking the intersection of the optical axis of the microscope lens and the CCD target surface as the origin of the coordinate system, its coordinate system is P in Cartesian space. wo (28) The coordinate system of the second microscopic vision system, P, remains parallel and in the same direction in both the X and Y directions. w2 (22) Taking the intersection of the optical axis of the microscope lens and the CCD target surface as the origin of the coordinate system, its coordinate system is P in Cartesian space. wo (28) The third microscopic vision system (3) is installed at the end of the robotic arm (6) and is parallel and in the same direction in both the X and Y directions. The coordinate system of the third microscopic vision system is P. w3 (23) Taking the intersection of the optical axis of the microscope lens and the CCD target surface as the origin of the coordinate system, and the coordinate system P at the end of the robotic arm as the origin of the coordinate system. w6 (26) Keep parallel in the Z-axis direction, the fourth microscopic vision system (4) is installed on the horizontal plane, the coordinate system of the fourth microscopic vision system P w4 (24) Taking the intersection of the optical axis of the microscope lens and the CCD target surface as the origin of the coordinate system, and the coordinate system P at the end of the robotic arm as the origin of the coordinate system. w6 (26) Keep parallel in the Z-axis direction; Step S2: The image offset matrix of the first microscopic vision system (1) is obtained by actively moving the first microscopic vision system (1) and the second microscopic vision system (2) respectively. Image offset matrix of the second microscopic vision system (2) Calibration; Step S3: Calibrate the image Jacobian matrix of the position and motion control of the first microscopic vision system (1) and the second microscopic vision system (2) to determine the position and motion control of part A (7) and part B (8) in the robot arm base coordinate system P. w (27) X, Y, Z axis position deviations d X d Y d Z Calculation; Step S4; calibrate the image Jacobian matrix of the angle motion control of the first microscopic vision system (1) and the second microscopic vision system (2) to determine the angle deviation of part A (7) and part B (8) around the X-axis and Y-axis. , Calculation; Step S5: Calibrate the relationship matrix between the third microscopic vision system (3) and the fourth microscopic vision system (4) to determine the rotation angle deviation between part A (7) and part B (8). calculate.

2. The calibration method for a microscopic visual inspection device assembled using multi-scale micro / nano devices as described in claim 1, characterized in that, Calculate according to the following formula and ： The image offset matrix of the first microscopic vision system (1) is realized by the active motion of the first microscopic vision system (1) and the second microscopic vision system (2) respectively. Image offset matrix with the second microscopic vision system The calibration of the image offset matrix describes the offset of the image points caused by the linear movement of the microscopic vision system along the focusing axis. The calibration object is fixed in the field of view of the first microscopic vision system (1). The first microscopic vision system (1) moves along the focusing axis n times, n≥3, and obtains n+1 known positions. Then the relationship shown in formula (1) exists. (1) Calculated using the linear least squares method As shown in formula (2), the image offset matrix of the second microscopic vision system (2) Adopted and The same calibration method can be used to obtain: (2) Where n is the number of times the first microscopic vision system (1) moves along the focusing axis; ,i=1,2,...n is the motion change of the first microscopic visual system (1); , ) represents the change in image coordinates of the calibrator between the first microscopic vision system (1) and the second microscopic vision system (2).

3. The calibration method for a microscopic visual inspection device assembled using multi-scale micro / nano devices as described in claim 2, characterized in that, The image Jacobian matrix for calibrating the position and motion control of the first microscopic vision system (1) and the second microscopic vision system (2) realizes the positioning and motion control of parts A (7) and B (8) in the robot arm base coordinate system P. w (27) X, Y, Z axis position deviations d X d Y d Z The calculation; first, calibrate the image Jacobian matrix J of the translational motion change of the robotic arm (6) to the translational motion change in the image space. V Install the calibration object at the end of the robotic arm (6), and let the robotic arm (6) move along the base coordinate system P. w (27) Perform n movements, n≥3, and record the motion amount of the robotic arm (6) respectively. x i , y i , z i , i=1,2,...n, After each movement of the robotic arm (6), the first microscopic vision system (1) and the second microscopic vision system (2) change their motion along the focusing axis in order to focus the calibration object. and And the change in coordinates of the calibrated object in the first microscopic vision system (1) , ) and the coordinate change in the second microscopic vision system (2) , Then, the relationship between the change in image spatial motion and the change in Cartesian spatial motion is shown in formula (3); (3) Where X is the coordinate system of the robotic arm (6) along the base coordinate system P. w (27) The motion change matrix formed by the nth motion of the robotic arm, C is the motion change matrix formed by the robotic arm (6) along the base coordinate system P of the robotic arm. w The image position change matrix of the calibration object formed by the nth movement of (27) can be obtained by using the least squares method to obtain the image Jacobian matrix J. V As shown in formula (4) （4） Through J V The spatial positional deviation (d) between part A and part B is calculated. X d Y d Z As shown in formula (5), （5） in, The image offset matrix of the first microscopic vision system (1) is... The image offset matrix of the second microscopic vision system (2) The first microscopic vision system (1) focuses the motion changes of parts A (7) and B (8) along the focusing axis, respectively. The second microscopic vision system (2) focuses the motion changes of parts A (7) and B (8) along the focusing axis, respectively. , )and( , The image position deviations of parts A (7) and B (8) in the first microscopic vision system (1) and the second microscopic vision system (2) are respectively.

4. The calibration method for a microscopic visual inspection device assembled using multi-scale micro / nano devices as described in claim 3, characterized in that, The image Jacobian matrix of the first microscopic vision system (1) and the second microscopic vision system (2) is used to calibrate the angle motion control, thereby realizing the angle deviation of part A (7) and part B (8) around the X-axis and around the Y-axis. , The calculation first involves calibrating the image Jacobian matrix J from the angle change of the robotic arm (6) to the angle change in the image space. R The robotic arm (6) moves the calibration object around the robotic arm's base coordinate system P. w (27) Moves the robot arm along the X and Y axes n times, n≥3, and record the movement of the robot arm (6) around the robot arm base coordinate system P. w (27) The angular change between the X-axis and the Y-axis is α i and β i i=1,2,...n, and the angular change of the calibrated object in the first microscopic vision system (1). Angular change with the second microscopic vision system (2) Then the robotic arm (6) moves along the base coordinate system P of the robotic arm. w The relationship between the spatial motion change of (27) and the spatial motion change of the image is shown in formula (6); Calculate according to the following formula , : (6) Where χ represents the calibration object along the robot arm's base coordinate system P w (27) angular change matrix, To determine the nth angular change matrix of the calibration object in the image space, the image Jacobian matrix J in formula (6) can be obtained using the least squares method. R As shown in formula (7): (7) Through J R The spatial angular deviation between part A (7) and part B (8) was calculated. , As shown in formula (8), (8) in , The image angle deviations of parts A (7) and B (8) in the first microscopic vision system (1) and the second microscopic vision system (2) are respectively.

5. The calibration method for a microscopic visual inspection device assembled using multi-scale micro / nano devices as described in claim 4, characterized in that, Derivation of the coordinate system P of the third microscopic vision system w3 (23) and the coordinate system P of the robotic arm end effector w6 The rotation relation matrix R of (26) p The third microscopic vision system coordinate system P w3 (23) Rotate clockwise α around the Z-axis to align with the coordinate system P at the end of the robotic arm. w6 (26) If they are parallel, then part A (7) and part B (8) are parallel in the third microscopic vision system coordinate system P. w3 (23) Positional deviation ( , , Transform to the robot arm end-effector coordinate system P w6 (26) Positional deviation ( , , As shown in formula (9); (9) The calibration object is made clearly visible in the focal plane of the third microscopic vision system (3), and then the robotic arm (6) is controlled to move in the coordinate system P at the end of the robotic arm. w6 (26) Moves the robotic arm n times in the XY plane, where n≥3, and record the changes in motion of the robotic arm respectively. , ,0), i=1,2,...n, and the change in image coordinates of the calibrator in the third microscopic vision system (3) , Then, the relationship shown in formula (10) exists, where k w3 The image scaling factor of the third microscopic vision system (3); (10) R is calculated using the linear least squares method. p As shown in formula (11): (11) Derivation of the coordinate system P at the end effector of the robotic arm w6 (26) with the robot arm base coordinate system P w (27) Rotation relationship matrix, part A (7) and part B (8) in the coordinate system P of the end of the robot arm w6 (26) Positional deviation ( , , Transformed into the robot arm's base coordinate system P w (27) Positional deviation ( , , As shown in formula (12); In the current state, the coordinate system P of the robotic arm's end effector... w6 (26) First, rotate around the coordinate system P at the end of the robotic arm. w6 (26) The clockwise rotation angle of the X-axis Then, around the coordinate system P at the end of the robotic arm w6 (26) Y-axis clockwise rotation angle Finally, the coordinate system P at the end of the robotic arm is rotated. w6 (26) The clockwise rotation angle of the Z-axis Afterwards, the coordinate system P of the robotic arm is... w (27) Parallel; (12) in( , , Let P be the coordinate system of part A (7) and part B (8) at the end of the robotic arm. w6 (26) positional deviation, ( , , )for( , , Transformed into the robot arm's base coordinate system P w (27) positional deviation, , , It can be read directly from the robotic arm's teach pendant without the need for recalibration; Derivation of the coordinate system P of the fourth microscopic vision system w4 (24) with the robot arm base coordinate system P w The rotation relation matrix R of (27) o Since both the fourth microscopic vision system (4) and the robotic arm (6) are mounted on the base platform, and mechanical installation angle errors are not considered, the coordinate system P of the fourth microscopic vision system is... w4 (24) Rotate clockwise around the Z-axis With the robot arm base coordinate system P w (27) Parallel, part A (7) and part B (8) are in the fourth microscopic vision system coordinate system P w4 (24) Positional deviation ( , , Transform to the robot arm's base coordinate system P w The positional deviation of (27) is shown in formula (13): (13) in( , , Let P be the coordinate system of part A (7) and part B (8) in the fourth microscopic vision system. w4 (24) Positional deviation; The calibration object is made clearly visible in the focal plane of the fourth microscopic vision system (4), and then the robotic arm (6) is controlled to move in the robotic arm base coordinate system P. w (27) Moves n times in the XY plane, n≥3, and record the change in motion of the robotic arm (6) respectively. , ,0), i=1,2,...n, and the change in image coordinates of the calibrator in the fourth microscopic vision system (4) , Then, the relationship shown in formula (14) exists, where k w4 The image scaling factor of the fourth microscopic vision system (4); (14) R is calculated using the linear least squares method. o As shown in formula (15); (15) By deriving formulas (9), (12), and (13), the coordinate system P of the third microscopic vision system is obtained. w3 (23) with the fourth microscopic vision system coordinate system P w4 The rotation matrix R of (24) m Part A (7) and Part B (8) are in the third microscopic vision system coordinate system P w3 (23) The positional deviation is transformed into the fourth microscopic vision system coordinate system P. w4 The positional deviation of (24) is shown in formula (16); (16) The coordinate system P of the robotic arm's end effector is realized through motion control. w6 (26) with the robot arm base coordinate system P w (27) In the robot arm base coordinate system P w (27) is parallel to the XY plane. Ignoring mechanical installation errors, the part A (7) at the end of the robotic arm is parallel to the base coordinate system P of the robotic arm. w (27) The XY plane is parallel. Since the operating table is also installed on the base platform, the mechanical installation angle error is not considered. The part B (8) on the operating table is parallel to the base coordinate system P of the robot arm. w (27) If the XY planes are parallel, then part B (8) lies in the third microscopic vision system coordinate system P. w3 The orientation of (23) around the Z-axis is converted into the coordinate system P of the fourth microscopic vision system. w4 The orientation of (24) around the Z-axis is shown in formula (17): (17) in For part B (8) in the third microscopic vision system coordinate system P w3 (23) attitude around the Z-axis The transformed coordinates in the fourth microscopic vision system P w4 (24) The orientation of the z-axis; (18) in For part A (7) in the fourth microscopic vision system coordinate system P w4 (24) Attitude detection around the Z-axis, The rotation angle deviation between part A (7) and part B (8) is given.

Images