A calibration method for a three-dimensional microscope imaging system
Through the multi-step calibration method of the three-dimensional microscope imaging system, including initial position, rotation axis, translation axis and calibration plate height calibration, the accuracy problems caused by mechanical scanning module errors are solved and the accuracy of three-dimensional microscopy imaging is improved.
Patent Information
- Application Number
- CN202411191320.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-28
AI Technical Summary
Due to the installation error of the mechanical scanning module and the control motion error of the three-dimensional microscopic imaging system, the hand and eye matrix is inaccurate, making it difficult to meet the accuracy requirements of three-dimensional microscopic imaging.
A calibration method of a three-dimensional microscope imaging system is adopted, including initial posture calibration, rotation axis calibration, translation axis calibration, offset calibration and calibration plate height calibration. By identifying the characteristic points of the calibration plate and the multi-view beam adjustment method, the adjustment of Camera position and posture, compute and compensate for rotation and translation errors, forming a clear image of a single grid.
Through the calibration of the internal and external parameters of the far-field correction optical system and the calibration of the mechanical scanning module, the spatial accuracy of the three-dimensional microscope is improved and the accuracy requirements of the three-dimensional microscope are met.
Smart Images

Figure CN118730906B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of biomedical sample detection instruments, and in particular to a calibration method for a three-dimensional microscope imaging system. Background Art
[0002] The three-dimensional microscopic imaging system includes a hardware part and a software part. The hardware part includes a sample morphology measurement module, a microscopic imaging module, a three-dimensional mechanical scanning module, and a sample adsorption module; the software part includes a three-dimensional image reconstruction module and an image evaluation feedback module. The three-dimensional mechanical scanning module includes a translation system, a rotation system, and a normal linear module. The translation system drives the sample and the microscopic imaging module to move in the horizontal and vertical directions; the rotation system drives the sample to be tested and the microscopic imaging module to rotate within the plane; the normal linear module drives the microscopic imaging module to move along the normal direction of the grid unit on the sample surface. The module performs regional and height-divided microscopic imaging of the entire surface of the sample by moving the microscopic imaging module in a fixed-point normal direction so as to translate and rotate the microscopic imaging module in three-dimensional space.
[0003] At present, there are inevitable errors in the 3D imaging system. According to various materials, during the installation process, the various connectors of the mechanical scanning module will cause the axes of the two rotating shafts to be unable to coincide at the origin, which will cause actual eccentricity, causing the center of the camera that originally remained stationary to move when the rotating shaft rotates, resulting in errors; the processing process of the mechanical connector and the deformation caused by its distal deflection will cause errors; the 3D scanner projects a sinusoidal fringe pattern, and there are errors in the process of reconstructing the point cloud after identifying the feature points on the calibration plate; the normal calculation method of the point cloud is through the local tangent plane fitting method, which is different from the actual normal; the calibration plate is made by laser processing, and there are manufacturing errors. In summary, the mechanical scanning module is the main source of error in the 3D imaging system.
[0004] Therefore, due to factors such as installation errors and robot arm control motion errors, the hand-eye matrix calculated by mechanical design of the three-dimensional microscopic imaging system is often inaccurate. Calibration is required to meet the accuracy requirements of three-dimensional microscopic imaging. Summary of the invention
[0005] In view of the lack of three-dimensional microscope calibration methods, the present invention provides a calibration method for a three-dimensional microscope imaging system. The technical solution adopted by the present invention is:
[0006] A calibration method for a three-dimensional microscope imaging system, the three-dimensional microscope imaging system comprising a visual system and a mechanical scanning module, the visual system being a three-dimensional microscope, the mechanical scanning module being a six-axis mechanical platform with three translational degrees of freedom and three rotational degrees of freedom, the six-axis mechanical platform carrying the microscope moving to any spatial position in any posture, the mechanical scanning module being used to complete the posture adjustment task and continuously shoot a grid along the normal direction, complete the depth of field synthesis task, and form a clear image of a single grid, the method comprising the following steps:
[0007] Step 1: Calibrate the initial position of the microscope; make sure that the imaging plane of the microscope is parallel to the plane of the calibration plate in the initial position;
[0008] Step 2: Calibrate the rotation axis: calibrate the rotation axis at each angle combination using a calibration plate, calculate the rotation error and compensate for it;
[0009] Step 3: Translation axis calibration: The translation axis is calibrated by calculating the angle between the calibration plate and the mechanical scanning module;
[0010] Step 4: Offset calibration: When scanning an object with height, the mechanical scanning module performs offset compensation in the xy plane according to the height;
[0011] Step 5: Calibrate the height of the calibration plate; compensate for the height unevenness caused by the tilt of the calibration plate.
[0012] Furthermore, the step 1 includes the following specific steps:
[0013] Step 1.1: Move the microscope to the initial position of the mechanical scanning module, so that the camera plane is roughly parallel to the calibration plate plane in the initial position, and use this position as the rough zero position;
[0014] Step 1.2: Perform camera calibration by identifying the feature points of the calibration plate, and obtain the actual magnification m of the camera under multiple postures, thereby determining the intrinsic parameter matrix of the camera;
[0015] Step 1.3: Solve the camera extrinsic matrix at the coarse zero position. The camera extrinsic matrix represents the spatial position relationship between the camera plane and the calibration plate plane. Adjust the camera position and posture according to the extrinsic matrix information, and repeat steps 1.1 to 1.3 multiple times to achieve the parallel accuracy requirements of the two planes.
[0016] Furthermore, the step 2 comprises:
[0017] Step 2.1: Compensate for the error of the rotational freedom around the x-axis and the error of the rotational freedom around the y-axis;
[0018] Step 2.2: Compensate for the error between the true rotation axis and the assumed rotation center.
[0019] Furthermore, the step 2.1 includes the following specific steps:
[0020] Step 2.1.1: Rotate the x-axis and y-axis at fixed angle intervals in a quadrant, and take a picture of the calibration plate at each angle combination. The center of the calibration plate needs to be photographed;
[0021] Step 2.1.2: Mark the pixel coordinates of the center point in each image, and solve the difference between the pixel coordinates and the center pixel coordinates, and use the rotation axis angle at this time to calculate the ideal extrinsic parameter matrix under this posture. The extrinsic parameter matrix is not completely consistent with the real extrinsic parameter matrix, but meets the accuracy requirements of the three-dimensional microscope system;
[0022] Step 2.1.3: The external parameter matrix is combined with the internal parameter matrix solved in step 1 to form a homography matrix under the posture, so as to solve the real deviation value through the pixel deviation value;
[0023] Step 2.1.4: A set of x offsets and y offsets can be calculated for each rotation angle combination. The x offsets and y offsets required for each angle in the entire quadrant space are fitted and compensated using the linear multi-segment difference method.
[0024] Furthermore, the step 2.2 includes the following specific steps:
[0025] Step 2.2.1: Move the microscope to the initial zero position and align the center of the field of view with the center point pattern of the calibration plate;
[0026] Step 2.2.2: Control the base to rotate clockwise at a fixed angle and photograph the calibration plate, marking the pixel coordinates of the center pattern of the calibration plate in each photo;
[0027] Step 2.2.3: Fit these coordinates into a circle, the center of the circle is the rotation center of the base, and the radius is the offset.
[0028] Furthermore, the step 3 includes the following specific steps:
[0029] Step 3.1: Control the microscope to translate only along the x-axis or only along the y-axis, and the translation distance is the side length of the chessboard;
[0030] Step 3.2: After each move, record the pixel coordinates of the corner points of the chessboard within the field of view;
[0031] Step 3.3: After moving to the border of the chessboard, fit all recorded points into an oblique line. The angle between this oblique line and the x-axis or y-axis is the required angle.
[0032] Furthermore, the step 4 includes the following specific steps:
[0033] Step 4.1: Raise the calibration plate to a fixed height, rotate the x-axis and y-axis at fixed angle intervals in a quadrant, and photograph the calibration plate at each angle combination. The center of the calibration plate needs to be photographed;
[0034] Step 4.2: Mark the pixel coordinates of the center point in each image, and calculate the difference between the pixel coordinates and the center pixel coordinates;
[0035] Step 4.3: Using the rotation axis angle at this time, an ideal extrinsic parameter matrix is obtained for the position and posture. The extrinsic parameter matrix is not completely consistent with the real extrinsic parameter matrix but meets the accuracy requirements of the three-dimensional microscope system;
[0036] Step 4.4: The external parameter matrix is combined with the internal parameter matrix solved in step 1 to form a homography matrix under the posture, so as to solve the real deviation value through the pixel deviation value;
[0037] Step 4.5: A set of x offsets and y offsets can be calculated for each rotation angle combination. The x offsets and y offsets required for each angle in the entire quadrant space are fitted and compensated using a linear multi-segment difference method.
[0038] Furthermore, the step 5 includes the following specific steps:
[0039] Step 5.1: Take photos at multiple corner points of the calibration plate with a certain tilt posture, and calculate the error between the corner point and the center of the field of view at each position;
[0040] Step 5.2: Substitute the error value into the height calibration algorithm to inversely calculate the required height value, and use the xy plane coordinates of each corner point and the inverse height to fit the calibration plate plane equation to complete the calibration of the calibration plate height.
[0041] Furthermore, the mechanical scanning module is a six-axis robotic arm. Under six degrees of freedom, the six-axis robotic arm carries a microscope and moves to any spatial position in any posture. Step 1 is the camera calibration of the microscope, including the following specific steps:
[0042] Step 1.1: Move the microscope to the initial position of the six-axis robot arm. In this position, the camera plane is roughly parallel to the calibration plate plane. This position is used as the rough zero position.
[0043] Step 1.2: Calibrate the camera by identifying the feature points of the calibration plate, and use the multi-view bundle adjustment method to obtain the actual camera magnification in multiple positions m , thereby determining the intrinsic parameter matrix of the camera, adjusting the posture of the microscope by changing the multiple rotation axes of the six-axis robot, and photographing a series of calibration plate patterns in different spatial planes;
[0044] Step 1.3: Solve the camera extrinsic matrix at the coarse zero position. The camera extrinsic matrix represents the spatial position relationship between the camera plane and the calibration plate plane. Adjust the camera position and posture according to the extrinsic matrix information. Repeat steps 1.1 to 1.3 multiple times to achieve the parallel accuracy requirements of the two planes. Complete the camera calibration of the microscope and consider all the motor positions of the six-axis robot arm to be zero at this time.
[0045] Furthermore, step 2 and step 3 are for degree of freedom calibration, which specifically include the following steps:
[0046] Step 2: Translational DOF calibration;
[0047] First, the translational degrees of freedom in the system are calibrated, including the following steps:
[0048] Step 2.1: The six-axis robot controls the microscope to translate along a certain translational degree of freedom at a zero position;
[0049] Step 2.2: After each move, record the pixel coordinates of the marker points on the chessboard within the field of view;
[0050] Step 2.3: After moving to the border of the chessboard, all recorded points are fitted into an oblique line. The angle between this oblique line and the coordinate axis is the offset angle of the translational freedom. The offset caused by the offset angle is compensated for each movement.
[0051] Step 3: Rotational freedom calibration;
[0052] Step 3.1: The six-axis robot controls the microscope to rotate along a certain rotational degree of freedom at the zero position, photographs the chessboard in a series of postures, and completes depth of field synthesis and landmark recognition;
[0053] Step 3.2: The exact real magnification obtained by solving m , determine the world coordinates of the pose according to the intrinsic parameter matrix, and derive the extrinsic parameter matrix from the landmarks;
[0054] Step 3.3: Use the external parameter matrix to find the actual posture under this posture.
[0055] The beneficial effects of adopting the above technical method are:
[0056] A calibration method for a three-dimensional microscope imaging system of the present invention meets the spatial accuracy requirements of a three-dimensional microscope and improves the accuracy of the three-dimensional microscope imaging system through calibration of internal and external parameters of a far-field correction optical system, calibration of a mechanical scanning module, unification of the coordinates of the mechanical scanning module and the coordinates of a three-dimensional scanner, and a design of a checkerboard calibration plate for spatial calibration. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1This is the feature point recognition diagram in the camera calibration during the initial position calibration of the microscope;
[0058] Figure 2 This is the data diagram of the offset between the origin and the center of the camera's field of view during rotation axis calibration;
[0059] Figure 3 This is a schematic diagram of the calibration of the rotation axis of the sample stage during the rotation axis calibration;
[0060] Figure 4 Schematic diagram of the difference between the x-axis and y-axis of the calibration plate and the x-axis and y-axis of the actual mechanical scanning platform;
[0061] Figure 5 Design drawing of the calibration board. DETAILED DESCRIPTION
[0062] The specific implementation modes of the present invention are described in detail below with reference to the accompanying drawings. Example
[0063] This embodiment proposes a method for calibrating a three-dimensional scanning microscope based on the system constructed by the applicant's previous patent (ZL201911111456.4). As the main application scenario of the invention, the three-dimensional microscope imaging system of the present invention includes a visual system and a mechanical scanning module. The visual system is a three-dimensional microscope, and the mechanical scanning module is a six-axis mechanical platform with three translational degrees of freedom and three rotational degrees of freedom built by the applicant. The six-axis mechanical platform can carry the microscope to move to any spatial position in any posture. In addition to completing the posture adjustment task, the mechanical scanning module can also continuously shoot a grid along the normal direction to complete the depth of field synthesis task and form a clear image of a single grid.
[0064] A calibration method for a three-dimensional microscope imaging system of the present invention comprises the following steps:
[0065] Step 1: Calibrate the initial position of the microscope; make sure that the imaging plane of the microscope is parallel to the plane of the calibration plate in the initial position;
[0066] The optical system of the 3D microscope system is an infinite optical system, and its internal reference model is a telecentric model, not a common pinhole model. Its mathematical expression is as follows:
[0067] A = ; B = ;H = A * B;
[0068] Where A is the camera's intrinsic parameter matrix, B is the camera's extrinsic parameter matrix, and H is the homography matrix. m is the actual magnification, du is the pixel size of the camera, r11 -r23 represents the camera posture, tx and ty Represents the spatial position of the camera.
[0069] Step 1.1: Move the microscope to the initial position of the mechanical scanning module. In this position, the camera plane can be roughly parallel to the calibration plate plane by naked eyes or a level. This position is used as the rough zero position.
[0070] Step 1.2: Calibrate the camera by identifying the feature points of the calibration plate and obtain the actual magnification of the camera in multiple positions m , thereby determining the camera's intrinsic parameter matrix;
[0071] Step 1.3: Solve the camera extrinsic matrix at the coarse zero position. The camera extrinsic matrix represents the spatial position relationship between the camera plane and the calibration plate plane. Adjust the camera position and posture according to the extrinsic matrix information, and repeat steps 1.1 to 1.3 multiple times to achieve the parallel accuracy requirements of the two planes.
[0072] like Figure 1 As shown in the figure, there are thirteen black marking points in the white square (square side length 4*4 mm) in the center of the calibration plate. These thirteen marking points can be divided into two parts: the upper half of the 3*4 dot matrix and the lower half of the origin as the center of the entire calibration plate. Based on the origin, the origin is 0.5 mm away from the edge of the black square below and equidistant from the black squares on both sides. The point below the dot matrix is 1.5 mm away from the origin longitudinally and is located in the center horizontally. The dot spacing between the dot matrices is 0.5 mm. The white dots are the points obtained by back-projection using the obtained internal and external parameter matrix. It can be seen that the back-projection effect is good, proving that the obtained internal and external parameter matrix is accurate enough.
[0073] Step 2: Calibrate the rotation axis: calibrate the rotation axis at each angle combination using a calibration plate, calculate the rotation error and compensate for it;
[0074] Due to machining errors, motor rotation errors, gravity errors and other factors, the center of the field of view often shifts after rotation before the rotation axis is calibrated. Figure 2 As shown in the figure, the x-axis represents the angle of the first rotation axis; the y-axis represents the angle of the second rotation axis, and the vertical axis represents the offset between the origin and the center of the camera's field of view. It can be found that without calibration, the error gradually increases with the increase of the degree of freedom of the rotation axis and the increase of the rotation angle of the rotation axis. Therefore, it is necessary to compensate for this type of rotation error. Rotation axis calibration includes:
[0075] Step 2.1: Compensate for the error of the rotational freedom around the x-axis and the error of the rotational freedom around the y-axis;
[0076] Step 2.1.1: Rotate the x-axis and y-axis at fixed angle intervals in a quadrant, and take a picture of the calibration plate at each angle combination. The center of the calibration plate needs to be photographed;
[0077] Step 2.1.2: Mark the pixel coordinates of the center point in each image, and solve the difference between the pixel coordinates and the center pixel coordinates. The ideal extrinsic matrix for this position can be obtained using the rotation axis angle at this time. The extrinsic matrix is not completely consistent with the actual extrinsic matrix, but meets the accuracy requirements of the three-dimensional microscope system;
[0078] Step 2.1.3: The external parameter matrix and the internal parameter matrix solved in step 1 can be combined into a homography matrix under the posture, so that the real deviation value can be solved by the pixel deviation value;
[0079] Step 2.1.4: A set of x-offset and y-offset can be obtained for each rotation angle combination, and the x-offset and y-offset required for each angle in the entire quadrant space are fitted and compensated using the linear multi-segment difference method. If there is an error in the external parameter matrix, this step can be repeated multiple times to increase accuracy.
[0080] Step 2.2: Compensate for the error between the true rotation axis and the assumed rotation center;
[0081] Due to factors such as calibration plate installation error and manufacturing error, the actual sample stage rotation axis does not coincide with the preset sample stage rotation axis. Figure 3 The diagram in the figure shows the calibration diagram of the sample stage rotation axis obtained by shooting at 5° intervals and fixing the camera. Use the recognition function in the opencv library to identify the trajectory of the center point and fit it into a circle. The center of the circle is the actual center of the calibration plate, and the radius is the offset. Their difference needs to be solved and compensated to the point cloud. The center point of the calibration plate is considered to be the center point pattern marked during production. The rotation center of the base needs to be fitted and solved, including the following steps:
[0082] Step 2.2.1: Move the microscope to the initial zero position and align the center of the field of view with the center point pattern of the calibration plate;
[0083] Step 2.2.2: Control the base to rotate clockwise at a fixed angle (usually 1°) and photograph the calibration plate, marking the pixel coordinates of the center pattern of the calibration plate in each photo;
[0084] Step 2.2.3: Fit these coordinates into a circle, the center of the circle is the rotation center of the base, and the radius is the offset.
[0085] Step 3: Translation axis calibration: The translation axis is calibrated by calculating the angle between the calibration plate and the mechanical scanning module;
[0086] Due to installation error, there is a slight angular error between the calibration plate and the translation axis in the xy plane. Figure 4 As shown, the x-axis represents the actual distance (mm), and the y-axis represents the offset (mm). There is a difference between the x-axis and y-axis of the calibration plate and the x-axis and y-axis of the actual mechanical scanning platform. This difference will be more obvious at a distance. Using the feature points on the calibration plate, take a picture every 4 mm (i.e., the side length of the chessboard), identify the feature points, record the coordinates, and fit them into a straight line using the least squares method. The slope of the straight line equation is the tangent value of the angle between the calibration plate and the mechanical scanning module. The angle solution method is as follows:
[0087] Step 3.1: Control the microscope to translate only along the x-axis or only along the y-axis, and the translation distance is the side length of the chessboard;
[0088] Step 3.2: After each move, record the pixel coordinates of the corner points of the chessboard within the field of view;
[0089] Step 3.3: After moving to the border of the chessboard, fit all recorded points into an oblique line. The angle between this oblique line and the x-axis or y-axis is the required angle.
[0090] The 3D scanner and mechanical scanning module have their own base coordinate systems, which are unified by a high-precision micro-calibration plate. This mainly includes offset calibration and calibration plate height calibration.
[0091] Step 4: Offset calibration. According to the design principle of the mechanical scanning module, the mechanical scanning module needs to perform offset compensation in the xy plane according to the height when scanning a height object. Theoretically, this offset is proportional to the current height of the microscope and proportional to the sine of the rotation angle. The theoretical value of the rotation angle is not consistent with the actual value, which causes errors in the compensation. The higher the height and the larger the angle, the greater the error. The method to calibrate this offset compensation is as follows:
[0092] Step 4.1: Raise the calibration plate to a fixed height, rotate the x-axis and y-axis at fixed angle intervals in a quadrant, and photograph the calibration plate at each angle combination. The center of the calibration plate needs to be photographed;
[0093] Step 4.2: Mark the pixel coordinates of the center point in each image, and calculate the difference between the pixel coordinates and the center pixel coordinates;
[0094] Step 4.3: The ideal extrinsic parameter matrix under this posture can be obtained by using the rotation axis angle at this time. The extrinsic parameter matrix is not completely consistent with the real extrinsic parameter matrix but meets the accuracy requirements of the three-dimensional microscope system;
[0095] Step 4.4: The external parameter matrix and the internal parameter matrix solved in step 1 can be combined into a homography matrix under the posture, so that the real deviation value can be solved through the pixel deviation value;
[0096] Step 4.5: A set of x offsets and y offsets can be calculated for each rotation angle combination. The x offsets and y offsets required for each angle in the entire quadrant space are fitted and compensated using a linear multi-segment difference method.
[0097] Step 5: Calibrate the height of the calibration plate; compensate for the uneven height caused by the tilt of the calibration plate. Due to the installation error of the calibration plate, its plane is not on the standard horizontal plane and is tilted, which leads to uneven height on the calibration plate plane. Different heights will cause errors, so the height of the calibration plate needs to be compensated. The calibration method is as follows:
[0098] Step 5.1: Take photos at multiple corner points of the calibration plate with a certain tilt posture, and calculate the error between the corner point and the center of the field of view at each position;
[0099] Step 5.2: Substitute the error value into the height calibration algorithm to inversely calculate the required height value, and use the xy plane coordinates of each corner point and the inverse height to fit the calibration plate plane equation to complete the calibration of the calibration plate height.
[0100] like Figure 5 As shown in the figure, the calibration plate design for calibration is mainly composed of a series of checkerboards, rings and thirteen marking points in the center. The side length of the checkerboard grid is 4 mm, the diameter of the ring is 2 mm, and the radius of the overall calibration plate is 20 mm. For the 3D scanner, the origin is placed at the center point. For the scanning platform, the center point is used as the reference for calibration and calibration. Through such a checkerboard, the coordinates of the two can be unified. Example
[0101] As another application scenario of the invention, the mechanical scanning module can also be a commercial six-axis robotic arm. Under six degrees of freedom, the six-axis robotic arm can carry the microscope and move to any spatial position in any posture, similar to a self-built six-axis mechanical platform. Under this structure, step 1 in embodiment 1 is the camera calibration of the microscope, including the following steps:
[0102] Step 1.1: Move the microscope to the initial position of the six-axis robot arm. In this position, the camera plane is roughly parallel to the calibration plate plane. This position is used as the rough zero position.
[0103] Step 1.2: Calibrate the camera by identifying the feature points of the calibration plate, and use the multi-view bundle adjustment method to obtain the actual camera magnification in multiple positions m, thereby determining the camera's intrinsic parameter matrix. By changing the multiple rotation axes of the six-axis robot to adjust the microscope's posture, a series of calibration plate patterns in different spatial planes are photographed. For postures with too large shooting angles, the camera cannot complete clear shooting at a certain depth of field height. In this case, the depth of field synthesis algorithm can be used to compensate for this;
[0104] Step 1.3: Solve the camera extrinsic matrix at the coarse zero position. The camera extrinsic matrix represents the spatial position relationship between the camera plane and the calibration plate plane. Adjust the camera position and posture according to the extrinsic matrix information. Repeat steps 1.1 to 1.3 multiple times to achieve the parallel accuracy requirements of the two planes. Complete the camera calibration of the microscope and consider all the motor positions of the six-axis robot arm to be zero at this time.
[0105] Under this structure, step 2 and step 3 in the first embodiment can be summarized as degree of freedom calibration, which includes translational degree of freedom calibration and rotational degree of freedom calibration. Specifically, the following steps are included:
[0106] Step 2: Translational DOF calibration;
[0107] The calibration difficulty of the translational degree of freedom is lower than that of the rotational degree of freedom, and in the system of this embodiment, the error of the uncalibrated translational degree of freedom is smaller than that of the rotational degree of freedom. Therefore, the translational degree of freedom in the system is first calibrated, including the following steps:
[0108] Step 2.1: The six-axis robot controls the microscope to translate along a certain translational degree of freedom at a zero position;
[0109] Step 2.2: After each move, record the pixel coordinates of the marker points on the chessboard within the field of view;
[0110] Step 2.3: After moving to the border of the chessboard, fit all recorded points into an oblique line. The angle between this oblique line and the coordinate axis is the offset angle of the translational freedom. The offset caused by the offset angle needs to be compensated for each movement.
[0111] Step 3: Rotational freedom calibration;
[0112] According to the required end displacement, inverse kinematics is used to solve the required change in degrees of freedom. Unlike the aforementioned forward kinematics, the change in degrees of freedom obtained by inverse kinematics is not unique and has multiple solutions. Moreover, its operation is nonlinear, and the amount of operation increases with the increase in degrees of freedom. It includes the following steps:
[0113] Step 3.1: The six-axis robot controls the microscope to rotate along a certain rotational degree of freedom at the zero position, photographs the chessboard in a series of postures, and completes depth of field synthesis and landmark recognition;
[0114] Step 3.2: The exact real magnification obtained by solving m , determine the world coordinates of the pose according to the intrinsic parameter matrix, and derive the extrinsic parameter matrix from the landmarks;
[0115] Step 3.3: The actual posture under this posture can be calculated through the external parameter matrix.
[0116] The other steps in this embodiment are the same as those in the first embodiment.
[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope limited by the claims of the present invention.
Claims
1. A calibration method for a three-dimensional microscope imaging system, wherein the three-dimensional microscope imaging system comprises a visual system and a mechanical scanning module, wherein the visual system is a three-dimensional microscope, and the mechanical scanning module is a six-axis mechanical platform with three translational degrees of freedom and three rotational degrees of freedom, wherein the six-axis mechanical platform carries the microscope and moves to any spatial position in any posture, and the mechanical scanning module is used to complete the posture adjustment task and continuously shoot a grid along the normal direction to complete the depth of field synthesis task and form a clear image of a single grid, wherein the method is characterized in that: The method comprises the following steps: Step 1: Calibrate the initial position of the microscope; make sure that the imaging plane of the microscope is parallel to the plane of the calibration plate in the initial position; Step 1.1: Move the microscope to the initial position of the mechanical scanning module, so that the camera plane is roughly parallel to the calibration plate plane in the initial position, and use this position as the rough zero position; Step 1.2: Perform camera calibration by identifying the landmarks on the calibration plate, and obtain the actual camera magnification m under multiple poses, thereby determining the camera's intrinsic parameter matrix; Step 1.3: Solve the camera extrinsic matrix at the coarse zero position. The camera extrinsic matrix represents the spatial position relationship between the camera plane and the calibration plate plane. Adjust the camera position and posture according to the extrinsic matrix information. Repeat steps 1.1 to 1.3 multiple times to achieve the parallel accuracy requirements of the two planes. Step 2: Calibrate the rotation axis: calibrate the rotation axis at each angle combination using a calibration plate, calculate the rotation error and compensate for it; Step 3: Translation axis calibration: The translation axis is calibrated by calculating the angle between the calibration plate and the mechanical scanning module; Step 4: Offset calibration: When scanning an object with height, the mechanical scanning module performs offset compensation in the xy plane according to the height; Step 5: Calibrate the height of the calibration plate; compensate for the uneven height caused by the tilt of the calibration plate; The calibration plate is composed of a series of checkerboards, circles and thirteen marking points in the center. The thirteen marking points are set in the white square in the center of the calibration plate. The thirteen marking points are divided into two parts: the upper half of the 3*4 dot matrix and the lower half of the origin as the center of the entire calibration plate.
2. The calibration method of a three-dimensional microscope imaging system according to claim 1, characterized in that: The step 2 comprises: Step 2.1: Compensate for the error of the rotational freedom around the x-axis and the error of the rotational freedom around the y-axis; Step 2.2: Compensate for the error between the true rotation axis and the assumed rotation center.
3. The calibration method of a three-dimensional microscope imaging system according to claim 2, characterized in that: The step 2.1 includes the following specific steps: Step 2.1.1: Rotate the x-axis and y-axis at fixed angle intervals in a quadrant, and take a picture of the calibration plate at each angle combination. The center of the calibration plate needs to be photographed; Step 2.1.2: Mark the pixel coordinates of the center point in each image, and solve the difference between the pixel coordinates and the center pixel coordinates, and use the rotation axis angle at this time to calculate the ideal extrinsic parameter matrix under this posture. The extrinsic parameter matrix is not completely consistent with the real extrinsic parameter matrix, but meets the accuracy requirements of the three-dimensional microscope system; Step 2.1.3: The external parameter matrix is combined with the internal parameter matrix solved in step 1 to form a homography matrix under the posture, so as to solve the real deviation value through the pixel deviation value; Step 2.1.4: A set of x offsets and y offsets can be calculated for each rotation angle combination. The x offsets and y offsets required for each angle in the entire quadrant space are fitted and compensated using the linear multi-segment difference method.
4. The calibration method of a three-dimensional microscope imaging system according to claim 2, characterized in that: The step 2.2 includes the following specific steps: Step 2.2.1: Move the microscope to the initial zero position and align the center of the field of view with the center point pattern of the calibration plate; Step 2.2.2: Control the base to rotate clockwise at a fixed angle and photograph the calibration plate, marking the pixel coordinates of the center pattern of the calibration plate in each photo; Step 2.2.3: Fit these coordinates into a circle, the center of the circle is the rotation center of the base, and the radius is the offset.
5. The calibration method of a three-dimensional microscope imaging system according to claim 1, characterized in that: The step 3 comprises the following specific steps: Step 3.1: Control the microscope to translate only along the x-axis or only along the y-axis, and the translation distance is the side length of the chessboard; Step 3.2: After each move, record the pixel coordinates of the corner points of the chessboard within the field of view; Step 3.3: After moving to the border of the chessboard, fit all recorded points into an oblique line. The angle between this oblique line and the x-axis or y-axis is the required angle.
6. The calibration method of a three-dimensional microscope imaging system according to claim 1, characterized in that: The step 4 comprises the following specific steps: Step 4.1: Raise the calibration plate to a fixed height, rotate the x-axis and y-axis at fixed angle intervals in a quadrant, and photograph the calibration plate at each angle combination. The center of the calibration plate needs to be photographed; Step 4.2: Mark the pixel coordinates of the center point in each image, and calculate the difference between the pixel coordinates and the center pixel coordinates; Step 4.3: Using the rotation axis angle at this time, an ideal extrinsic parameter matrix is obtained for the position and posture. The extrinsic parameter matrix is not completely consistent with the real extrinsic parameter matrix but meets the accuracy requirements of the three-dimensional microscope system; Step 4.4: The external parameter matrix is combined with the internal parameter matrix solved in step 1 to form a homography matrix under the posture, so as to solve the real deviation value through the pixel deviation value; Step 4.5: A set of x offsets and y offsets can be calculated for each rotation angle combination. The x offsets and y offsets required for each angle in the entire quadrant space are fitted and compensated using a linear multi-segment difference method.
7. The calibration method of a three-dimensional microscope imaging system according to claim 1, characterized in that: The step 5 comprises the following specific steps: Step 5.1: Take photos at multiple corner points of the calibration plate with a certain tilt posture, and calculate the error between the corner point and the center of the field of view at each position; Step 5.2: Substitute the error value into the height calibration algorithm to inversely calculate the required height value, and use the xy plane coordinates of each corner point and the inverse height to fit the calibration plate plane equation to complete the calibration of the calibration plate height.
8. A calibration method for a three-dimensional microscope imaging system, the three-dimensional microscope imaging system comprising a visual system and a mechanical scanning module, the visual system being a three-dimensional microscope, the mechanical scanning module being a six-axis robotic arm, the six-axis robotic arm carrying the microscope moving to any spatial position in any posture under six degrees of freedom, the mechanical scanning module being used to complete the posture adjustment task and continuously shoot a grid along the normal direction, complete the depth of field synthesis task, and form a clear image of a single grid, characterized in that: The method comprises the following steps: Step 1: Calibrate the microscope camera; make sure that the microscope imaging plane is parallel to the calibration plate plane in the initial position; Step 1.1: Move the microscope to the initial position of the six-axis robot arm. In this position, the camera plane is roughly parallel to the calibration plate plane. This position is used as the rough zero position. Step 1.2: Calibrate the camera by identifying the landmarks on the calibration plate, and use the multi-view bundle adjustment method to obtain the actual camera magnification in multiple positions. m , thereby determining the intrinsic parameter matrix of the camera, adjusting the posture of the microscope by changing multiple rotation axes of the six-axis robot, and photographing a series of calibration plate patterns in different spatial planes; Step 1.3: Solve the camera extrinsic matrix at the coarse zero position. The camera extrinsic matrix represents the spatial position relationship between the camera plane and the calibration plate plane. Adjust the camera position and posture according to the extrinsic matrix information. Repeat steps 1.1 to 1.3 multiple times to achieve the parallel accuracy requirements of the two planes. Complete the camera calibration of the microscope, and consider all the motor positions of the six-axis robot arm to be zero at this time. Step 2: Translational DOF calibration; First, the translational degrees of freedom in the system are calibrated, including the following steps: Step 2.1: The six-axis robot controls the microscope to translate along a certain translational degree of freedom at a zero position; Step 2.2: After each move, record the pixel coordinates of the marker points on the chessboard within the field of view; Step 2.3: After moving to the border of the chessboard, all recorded points are fitted into an oblique line. The angle between this oblique line and the coordinate axis is the offset angle of the translational freedom. The offset caused by the offset angle is compensated for each movement. Step 3: Rotational freedom calibration; Step 3.1: The six-axis robot controls the microscope to rotate along a certain rotational degree of freedom at the zero position, photographs the chessboard in a series of postures, and completes depth of field synthesis and landmark recognition; Step 3.2: The exact real magnification obtained by solving m , determine the world coordinates of the pose according to the intrinsic parameter matrix, and derive the extrinsic parameter matrix from the landmark points; Step 3.3: Obtain the actual posture under this posture through the external parameter matrix; Step 4: Offset calibration: When scanning an object with height, the mechanical scanning module performs offset compensation in the xy plane according to the height; Step 5: Calibrate the height of the calibration plate; compensate for the uneven height caused by the tilt of the calibration plate; The calibration plate is composed of a series of checkerboards, circles and thirteen marking points in the center. The thirteen marking points are set in the white square in the center of the calibration plate. The thirteen marking points are divided into two parts: the upper half of the 3*4 dot matrix and the lower half of the origin as the center of the entire calibration plate.
