Measuring microscope calibration method based on ISP chip

Abstract

The invention discloses a measuring microscope calibration method based on an ISP chip, which is used for correcting optical distortion and geometric distortion in microscope imaging and improving the measuring precision of a microscope, and comprises the following steps: configuring a microscope system and adjusting illumination conditions to ensure the uniformity of an imaging environment; a calibration circle image on the calibration plate is collected, and coordinate normalization is carried out; detecting the edge of the calibration circle by using an image processing algorithm and optimizing the edge to sub-pixel precision; calculating a theoretical circle center coordinate of the calibration circle, and optimizing the actually detected circle center position through a least square method; correcting radial distortion, eccentric distortion, thin prism distortion and perspective error in sequence; the error is further corrected through edge detection, finally, the correction result is verified, correction parameters are solidified into the ISP chip, and the real-time correction function is achieved; according to the method, the precision and reliability of the microscope system in micron-order measurement are improved, and the method is suitable for the fields of high-precision microscopic imaging and scientific research.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a measuring microscope calibration method based on an ISP chip. Background Art

[0002] Microscope precision measurement can measure dimensions at the micron level and has high requirements for optics and image quality. Due to factors such as the complexity of lens design and process level, the actual imaging system cannot strictly meet the central perspective projection model, resulting in optical distortion errors between the actual image of the object point on the camera imaging surface and the ideal image.

[0003] There are currently three mainstream camera calibration methods:

[0004] (1) Zhang Zhengyou calibration method, which is a calibration method that uses the homography matrix between images to linearly solve all parameters except the distortion coefficient. The camera parameters obtained by this method are highly accurate and stable, so it is the most common calibration algorithm at present; however, to obtain the homography matrix relationship between images, at least two images must be taken from positions with significantly different angles, which is not suitable for measurement microscope systems with small depth of field and high magnification.

[0005] (2) Tsai algorithm, also known as the two-step method. Assuming that the image is only affected by radial distortion, that is, assuming that the distortion exists only in the direction perpendicular to the camera imaging surface, a linear equation group of the camera's internal and external parameters is established based on this, and then all parameters are solved by the least squares method. This method is fast and accurate, and only requires taking one picture. However, once the angle between the calibration object and the imaging surface is less than 30°, the linear equation group established by this method will have a large singularity, resulting in inaccurate parameters.

[0006] (3) Calibration methods based on projective geometry principles: a. Self-calibration methods based on the epipolar geometric relationship between feature points. This method requires matching the feature points of objects between images and parsing their basic matrices to establish equations about the camera's internal and external parameters. In this type of method, in order to calculate the internal and external parameters, it is necessary to decompose the basic matrix or the essential matrix. Since the decomposition process is particularly sensitive to noise, the accuracy of this type of method is not high. It is usually used in surveillance cameras, which do not require high accuracy but do not require special calibration objects.

[0007] In summary, these methods are not suitable for the calibration of measuring microscope systems. Therefore, there is an urgent need for a measuring microscope calibration method that can accurately detect and correct these distortions to ensure the imaging accuracy and measurement reliability of the measuring microscope. Summary of the invention

[0008] Technical purpose: In view of the shortcomings of the existing measurement microscope system calibration, the present invention discloses a measurement microscope calibration method based on an ISP chip, which can solve the problem of reduced microscope measurement accuracy caused by optical radial distortion, eccentric distortion, thin prism distortion, etc.

[0009] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0010] A method for calibrating a measuring microscope based on an ISP chip, comprising:

[0011] Prepare a microscope system including a measuring microscope and an ISP chip and a calibration plate for calibration, wherein the calibration plate has a calibration circle with known geometric features for generating images;

[0012] Use an illuminance meter and a spectrum analyzer to measure the illumination and color temperature of the microscope light source, and adjust the illumination and color temperature of the light source to the preset values;

[0013] The calibration plate is fixed on the stage of the measuring microscope, multiple images containing the calibration circle are collected by the microscope system, the initial image coordinates of the calibration circle are extracted by image processing software, and the extracted coordinates are converted into normalized coordinates;

[0014] Based on the geometric structure of the calibration plate, the theoretical center coordinates of the calibration circle are calculated, and the difference between the actual detected center coordinates and the theoretical center coordinates is optimized by the least squares fitting method to obtain the accurate theoretical center coordinates;

[0015] Detect the radial distortion, eccentric distortion and thin prism distortion of the microscope system, and calculate the corresponding distortion amount;

[0016] The image is corrected for distortion using the calculated radial distortion, eccentric distortion and thin prism distortion;

[0017] Based on the central perspective projection relationship, a perspective projection model is constructed and a perspective transformation matrix is ​​calculated, and the perspective error is corrected through the perspective transformation matrix;

[0018] Analyze edge positioning deviation through edge detection algorithm, and correct edge detection error by adjusting detection parameters and algorithm;

[0019] The accuracy of the calibration results is verified by using standard scale objects, and the verified calibration parameters are fixed into the ISP chip to realize the real-time calibration function of the measuring microscope.

[0020] Preferably, the calibration plate is fixed on the stage of the measuring microscope, a plurality of images including the calibration circle are collected by the microscope system, the initial image coordinates of the calibration circle are extracted by image processing software, and the extracted coordinates are converted into normalized coordinates, which specifically includes the following steps:

[0021] The first step is to fix the calibration plate on the stage of the measuring microscope to ensure that the horizontal and vertical positions of the calibration plate are accurate to prevent image deviation caused by tilt or movement of the calibration plate.

[0022] Step 2: Use the ISP chip to control the measuring microscope camera and collect multiple calibration plate images;

[0023] The third step is to use the image processing system to extract the feature points in the calibration plate image and convert the extracted feature point coordinates into normalized coordinates;

[0024] The calculation formula for normalized coordinates is as follows:

[0025]

[0026] Among them, (c x , c y ) is the optical center, that is, the coordinate of the intersection of the optical axis and the image plane, (x d ,y d ) is the normalized image coordinate, and (x, y) represents the image coordinate.

[0027] Preferably, the calculation formula of the radial distortion is as follows:

[0028]

[0029] Among them, δ xr ,δ yr They represent the horizontal and vertical distortion caused by radial distortion, k is the radial distortion coefficient of the lens, O(x d ,y d ) 5 Represents about x d and d High-order small quantities of fifth order and above; (x d ,y d ) is the normalized image coordinate, and the relationship between it and the image coordinate (x, y) is

[0030]

[0031] Among them, (c x , c y ) is the optical center, that is, the coordinate of the intersection of the optical axis and the image plane.

[0032] Preferably, the calculation formula of the eccentric distortion is as follows:

[0033]

[0034] In the formula, δ xp ,δ ypThey represent the horizontal and vertical distortion caused by eccentric distortion respectively; p1 and p2 represent the eccentric distortion coefficients, O(x d ,y d ) 4 Represents about x d and d Higher-order small quantities of fourth order and above.

[0035] Preferably, the calculation formula of the thin prism distortion is as follows:

[0036]

[0037] In the formula, δ xs ,δ ys Respectively represent the horizontal and vertical distortion caused by thin prism distortion; s1 and s2 represent the thin prism distortion coefficient, O(x d ,y d ) 4 Represents about x d and d Higher-order small quantities of fourth order and above.

[0038] Preferably, the coordinates of the image are corrected using the calculated radial distortion, eccentric distortion and distortion in the horizontal and vertical directions caused by the thin prism distortion, and the calculation formula for the coordinates after the image correction is as follows:

[0039]

[0040] Among them, x p and p represents the coordinates of the image after distortion correction, δ xr ,δ yr They represent the horizontal and vertical distortion caused by radial distortion, δ xp ,δ yp They represent the horizontal and vertical distortion caused by eccentricity distortion, δ xs ,δ ys They represent the horizontal and vertical distortions caused by thin prism distortion.

[0041] Preferably, after converting the initial image coordinates of the calibration circle into normalized coordinates, the Harris corner detection algorithm can be used in the normalized coordinate image to detect the corner point positions in the calibration plate image, and the detected corner point coordinates are refined by a sub-pixel precision algorithm to improve the accuracy of calibration circle detection.

[0042] Preferably, the calculation formula of the theoretical center coordinates is:

[0043]

[0044] in and They represent the theoretical center coordinates of the calibration circle. The calibration circle adopts a 3*3 equal spacing layout. A r It means [x 3 y 3 …x 1 y 1 ], i and j represent the number of rows and columns of the calibration circle, respectively, a and b represent the theoretical center coordinates of the calibration circle in the middle area of ​​the calibration plate image, and k represents the slope value determined by the center coordinates of the calibration circle with three rows and three columns according to the principle of least squares method.

[0045] Preferably, a perspective projection model is constructed based on the central perspective projection relationship and a perspective transformation matrix is ​​calculated. The perspective error is corrected by the perspective transformation matrix. The specific calculation formula is as follows:

[0046]

[0047] From the above formula, we can get:

[0048]

[0049] Among them, x q ,y q 、z q represents the coordinates of the image after perspective error correction, x, y, and z represent the coordinates of the image without perspective error correction, z represents the distance from the center of the microscope objective lens to the CCD, which is a constant, α represents the angle of rotation around the x-axis when the image is not corrected for perspective error, and β represents the angle of rotation around the y-axis when the image is not corrected for perspective error.

[0050] Beneficial effects: The ISP chip-based measuring microscope calibration method provided by the present invention has the following beneficial effects:

[0051] 1. The present invention effectively eliminates geometric distortion in the optical system image by accurately correcting various optical distortions in the imaging of the measuring microscope, such as radial distortion, eccentric distortion, thin prism distortion and perspective error, so as to make the geometric shape of the image closer to the actual object, thereby significantly improving the imaging quality of the microscope; the present invention eliminates the image deformation problem caused by the non-parallelism of the imaging plane and the photographed object through perspective error correction, ensures that the geometric shape of the image is more in line with the actual situation, and ensures the accuracy of the measurement result.

[0052] 2. The present invention adopts advanced image processing technologies such as sub-pixel precision optimization algorithm and least square method to improve the detection accuracy of the calibration circle edge and theoretical circle center coordinates, ensuring the high precision and high reliability of the microscope in micron-level measurement; it is adaptable to a variety of optical configurations, and by solidifying the correction parameters into the ISP chip, the solution can achieve real-time correction under different optical configurations and usage conditions, ensuring the measurement accuracy and consistency of the microscope system in various environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0054] Figure 1 A flow chart of a method for calibrating a measuring microscope according to the present invention;

[0055] Figure 2 and Figure 3 This is a before and after comparison diagram of the measurement microscope calibrated by the present invention. DETAILED DESCRIPTION

[0056] The present invention will be described more clearly and completely below by way of a preferred embodiment in conjunction with the accompanying drawings, but the present invention is not limited to the scope of the embodiment.

[0057] like Figure 1 , a measurement microscope calibration method based on an ISP chip, comprising:

[0058] S101, prepare a microscope system including a measuring microscope and an ISP chip and a calibration plate for calibration, wherein the calibration plate has a calibration circle with known geometric features for generating an image.

[0059] In one embodiment, the calibration plate can be a dot matrix calibration plate or a glass slide checkerboard calibration plate, which has known geometric properties and is used to provide multiple reference points for calibration. When a glass slide checkerboard calibration plate is used, the grid spacing is 0.02 mm.

[0060] S102. Use an illuminance meter and a spectrum analyzer to respectively detect and measure the illumination and color temperature of the microscope light source, and adjust the illumination and color temperature of the light source to preset values ​​to ensure the stability and consistency of the imaging light source.

[0061] In one embodiment, the specific steps are as follows:

[0062] Step 1: Use an illuminance meter to test the illuminance at the working distance of the microscope objective lens, and adjust the light source so that the illuminance reaches a preset standard value, such as 600 lux, to ensure that the light intensity of the imaging environment is consistent;

[0063] Step 2: Use a spectrum analyzer to measure the color temperature of the current light source, and adjust the color temperature of the light source according to the measurement result to ensure the color consistency of the imaging light source, which helps to reduce the interference of different color lights on imaging;

[0064] Step 3: Place the color card under the objective lens of the measuring microscope to ensure clear imaging, and use the software analysis tool provided by the ISP chip to take a picture of the color card;

[0065] Step 4: Place the calibration plate under the objective lens of the measuring microscope to ensure clear imaging, and use the image processing system with integrated ISP chip to capture the image of the calibration plate;

[0066] Step 5: Calculate the distortion coefficient through the image processing system, save the calculation result in matrix form, and solidify it into the ISP chip code;

[0067] Step 6. Start the ISP chip, collect the image of the calibration board, and check the image effect.

[0068] In one embodiment, the color card is the CLASSIC NANO 24 color card, and the illuminance meter is the Model ZDS-10 illuminance meter.

[0069] S103, fixing the calibration plate on the stage of the measuring microscope, using the microscope system to collect multiple images containing the calibration circle, using image processing software to extract the initial image coordinates of the calibration circle, and converting the extracted coordinates into normalized coordinates.

[0070] In one embodiment, the specific implementation process is as follows:

[0071] The first step is to fix the calibration plate on the stage of the measuring microscope to ensure that the horizontal and vertical positions of the calibration plate are accurate to prevent image deviation caused by tilt or movement of the calibration plate.

[0072] The second step is to use the ISP chip to control the measuring microscope camera and collect multiple images containing the calibration circle to ensure that the clarity and contrast of each image are high enough for subsequent feature point detection; collecting multiple images can average out the noise and ensure that the calibration points can be accurately identified even in imperfect lighting conditions;

[0073] The third step is to use the image processing system to extract feature points in the calibration circle image and convert the extracted feature point coordinates into normalized coordinates. The normalization process converts the image coordinates into a standard coordinate system corresponding to the physical space coordinates, such as normalizing the pixel coordinates to the range of 0 to 1, which helps to eliminate the influence of image size and position on the calibration results.

[0074] The calculation formula for normalized coordinates is as follows:

[0075]

[0076] Among them, (c x , c y ) is the optical center, that is, the coordinate of the intersection of the optical axis and the image plane, (x d ,y d ) is the normalized image coordinate, and (x, y) represents the image coordinate.

[0077] In one embodiment, after the initial image coordinates of the calibration circle are converted into normalized coordinates, the Harris corner point detection algorithm can be used in the normalized coordinate image to detect the corner point positions in the calibration plate image, and the detected corner point coordinates are refined by a sub-pixel precision algorithm to improve the accuracy of calibration circle detection. The specific implementation process is as follows:

[0078] The first step is to import the collected calibration plate image into the image processing system to ensure that the image has been pre-processed, such as denoising, contrast enhancement, etc., to improve the effect of corner detection;

[0079] Step 2: According to the type of calibration plate, for example, if a checkerboard calibration plate is used, enter the number of rows and columns of the checkerboard calibration plate;

[0080] Step 3: Use the Harris corner detection algorithm to identify the corners in the calibration plate image. The Harris corner detection algorithm determines the position of the corners by calculating the response function of each pixel in the calibration plate image. These corners are usually the parts with the most dramatic grayscale changes in the image.

[0081] The fourth step is to use a sub-pixel precision algorithm to perform secondary optimization in a small range around the detected corner point to determine the exact position of the corner point. This refinement can accurately locate the corner point to the sub-pixel level, which is much smaller than the size of a pixel, thereby greatly improving the accuracy of calibration.

[0082] The precise calibration circle edge position is used to calculate the theoretical circle center coordinates and perform various distortion corrections. Sub-pixel-level precise positioning reduces the deviation caused by detection errors during the correction process and improves the accuracy of distortion correction.

[0083] S104, based on the geometric structure of the calibration plate, calculating the theoretical center coordinates of the calibration circle, and using the least squares method to fit and optimize the difference between the actually detected center coordinates and the theoretical center coordinates, to obtain accurate theoretical center coordinates for comparison with the actually detected center position.

[0084] In one embodiment, the specific implementation process is as follows:

[0085] The first step is to derive the theoretical coordinates of all calibration circles based on the geometric characteristics of the calibration plate.

[0086] For the dot matrix calibration plate, the calibration circle adopts a 3*3 equally spaced layout with a circle size of 0.01mm. The calibration plate is read by taking a photo with the ISP chip, and the center coordinates of the circle are automatically identified and extracted, and each center coordinate is connected into a straight line from left to right. If the calibration plate is placed horizontally, the line connecting the center of the circle from left to right should be perpendicular to the line connecting the center of the circle from top to bottom, and the deviation is calculated from this.

[0087] Since the determined calibration sampling point is the center of the calibration circle of the dot matrix calibration plate, according to the geometric characteristics of the dot matrix calibration plate, under ideal conditions, the distance between the centers of the calibration circles after imaging is equal, the centers of the circles in the same row or column are on the same straight line, and the straight lines in the row and column directions are perpendicular; therefore, the theoretical center coordinates of the calibration circle in the middle area of ​​each dot matrix calibration plate image are set to (a, b), and the slope k of the theoretical center coordinates in the same row is, then the theoretical center coordinates of all circles in the field of view are The calculation formula is:

[0088]

[0089] in and Respectively represent the theoretical center coordinates of all calibration circles within the field of view, A r It means [x 3 y 3 …x 1 y 1 ], i and j represent the number of rows and columns of the calibration circle, respectively, a and b represent the theoretical center coordinates of the calibration circle in the middle area of ​​the calibration plate image, and k represents the slope value determined by the center coordinates of the calibration circle with three rows and three columns according to the principle of least squares method.

[0090] According to the principle of least squares method, the slope value k of the coordinates of the center of the three rows and three columns of the calibration circle in the middle area of ​​the field of view is used. On this basis, the sum of the squares of the distances between the theoretical center and the actual center of the nine calibration circles in the middle area is guaranteed to be minimum, and the values ​​of a and b are obtained, thereby calculating the coordinates of all theoretical centers. All distortion coefficients established by theoretical values ​​and actual values ​​are calculated in this way.

[0091] The theoretical center coordinates are used as reference points to correct radial distortion, tangential distortion, and thin prism distortion, so that these corrections can be based on the center of the circle to ensure the accuracy of the correction.

[0092] The second step is to use the least squares method to fit the error between the theoretical center coordinates and the actual detected center coordinates so that the sum of squared errors is minimized. The best fit is found by adjusting the parameters of the theoretical model, such as translation, scaling, and rotation, to correct any systematic deviations.

[0093] S105, detecting radial distortion in the image, and calculating the radial distortion amount caused by radial distortion. Radial distortion is the most important microscope lens distortion. This deformation will cause the image point to move radially. The farther away from the center of the image, the greater the deformation. Radial distortion is an error caused by the loss of similarity between the object and the image due to the non-constant magnification on the image plane of a pair of conjugate objects. The change in the focal length of the microscope lens has a greater impact on this distortion.

[0094] The calculation formula of the radial distortion is as follows:

[0095]

[0096] Among them, δ xr ,δ yr They represent the horizontal and vertical distortion caused by radial distortion, q is the radial distortion coefficient of the lens, O(x d ,y d ) 5 Represents about x d and d High-order small quantities of fifth order and above, such as (x d ,y d ) is the normalized image coordinate, (x, y) is the image coordinate.

[0097] By analyzing the center position offset in the image of the calibration plate, the eccentric distortion caused by the non-collinearity of the lens optical axis is determined, and the eccentric distortion amount of each image point is calculated. Due to assembly errors, the optical axes of multiple lenses of the component lens cannot be completely collinear, which causes eccentric distortion. This deformation is composed of radial distortion and tangential distortion.

[0098] The calculation formula of the eccentric distortion is as follows:

[0099]

[0100] In the formula, δ xp ,δ yp They represent the horizontal and vertical distortion caused by eccentric distortion respectively; p1 and p2 represent the eccentric distortion coefficients, O(x d ,y d ) 4 Represents about x d and d Higher-order small quantities of fourth order and above.

[0101] Analyze the image distortion caused by the thin prism effect, and calculate the thin prism distortion caused by the thin prism distortion in the image. The thin prism distortion is the image deformation caused by the manufacturing error of the optical lens of the measuring microscope and the manufacturing error of the CCD, which causes the additional radial distortion and tangential distortion. This distortion can be modeled by adding a thin prism to the optical system to cause the additional radial distortion and tangential distortion.

[0102] The calculation formula of the thin prism distortion is as follows:

[0103]

[0104] In the formula, δ xs ,δ ys Respectively represent the horizontal and vertical distortion caused by thin prism distortion; s1 and s2 represent the thin prism distortion coefficient, O(x d ,y d ) 4 Represents about x d and d Higher-order small quantities of fourth order and above.

[0105] S106, using the calculated radial distortion, eccentric distortion, and horizontal and vertical distortion caused by thin prism distortion to correct the coordinates of each image point, to obtain the corrected coordinates of each image point, the calculation formula is as follows:

[0106]

[0107] Among them, x p and p represents the coordinates of the image after distortion correction, δ xr ,δ yr They represent the horizontal and vertical distortion caused by radial distortion, δ xp ,δ yp They represent the horizontal and vertical distortion caused by eccentricity distortion, δ xs ,δ ys They represent the horizontal and vertical distortions caused by thin prism distortion.

[0108] S107, constructing a perspective projection model based on the central perspective projection relationship and calculating a perspective transformation matrix, and correcting perspective errors through the perspective transformation matrix.

[0109] According to the central perspective projection relationship, the image formed by a plane that is not perpendicular to the optical axis after passing through the lens is still a plane that is not perpendicular to the optical axis, but it can only be imaged on the image sensor that is perpendicular to the optical axis, thus causing perspective error. According to the central perspective projection relationship, the image formed by a tilted regular quadrilateral target should be ABCD (assuming that the plane coordinate system it is in is X2OY2), and the projection of this image on the CCD is abcd (assuming that the plane coordinate system it is in is X1OY1). Obviously, abcd is no longer a regular quadrilateral; if the distorted image on X1OY1 is to be restored to the ideal image on X2OY2, the coordinates of the ideal image plane coordinate system need to be converted into the coordinates of the X1Y1Z coordinate system. Since the origins of X2OY2 and X1OY1 coincide, it can be assumed that the coordinate plane X2OY2 is first rotated by an angle α around the X1 axis of the X1Y1Z coordinate system, and then rotated by an angle β around the Y1 axis.

[0110] The calculation formula is as follows:

[0111]

[0112] From the above formula, we can get:

[0113]

[0114] Among them, x q ,y q 、z q represents the coordinates of the image after perspective error correction, x, y, and z represent the coordinates of the image without perspective error correction, z represents the distance from the center of the microscope objective lens to the CCD, which is a constant, α represents the angle of rotation around the x-axis when the image is not corrected for perspective error, and β represents the angle of rotation around the y-axis when the image is not corrected for perspective error.

[0115] S108, analyzing edge positioning deviations through edge detection algorithms, and correcting edge detection errors by adjusting detection parameters and algorithms.

[0116] There are three illumination modes for measuring microscopes: bottom transmitted light illumination, ring light illumination, and coaxial light illumination. Due to the uneven illumination of the illumination system, the vignetting caused by the interaction between light and objects, and filtering, the edge may be offset. In the case of high edge positioning accuracy, there is a difference of Δ between the real edge and the extracted edge. Establish an image point (x e ,y e ) and the actual image point (x, y) is

[0117]

[0118] In the formula, Δ x , Δ yThey are the offsets in the x and y directions respectively.

[0119] Since the grayscale value of an image is directly related to the intensity of the light source, the intensity of the light source can be measured by calculating the average value of the center of the background image under different light source intensities. The quantified grayscale value is 0-255. The closer the grayscale value is to 255, the higher the light source intensity is. When the grayscale value reaches 255, the image is saturated, and the light source intensity level of the image is determined. Taking a calibration circle in the calibration plate as an example, when the light source intensity is undersaturated, the image is blurry, which is not conducive to edge feature extraction and image measurement accuracy. When the light source intensity is oversaturated, the image edge is clear, which is conducive to accurate edge positioning. However, since the grayscale value of the image background no longer changes after the light source intensity reaches a certain level, and the foreground grayscale value increases with the increase of the light source intensity, the edge is offset from the background to the foreground along the normal direction, resulting in edge position error. The image edge position error caused by different light source intensities can be expressed as (D is the theoretical calibration circle diameter, d is the actual calibration circle diameter), thus establishing the functional relationship between the light source intensity and the image edge position error, and thus calculating Δ x , Δ y , which is the difference between the actual coordinates and the coordinates measured by the image.

[0120] S109, verifying the accuracy of the calibration result by using a standard scale object, and fixing the verified calibration parameters into the ISP chip to realize the real-time calibration function of the measuring microscope.

[0121] like Figure 2 and Figure 3 As shown, the uncalibrated image before correction and the calibrated image after correction obtained by the camera acquisition lens are respectively given. It can be seen that the calibrated image after correction is clearer and the geometric shape of the image is more consistent with the actual situation.

[0122] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A calibration method for a measuring microscope based on an ISP chip, characterized in that: include: Prepare a microscope system including a measuring microscope and an ISP chip and a calibration plate for calibration, wherein the calibration plate has a calibration circle with known geometric features for generating images; Use an illuminance meter and a spectrum analyzer to measure the illumination and color temperature of the microscope light source, and adjust the illumination and color temperature of the light source to the preset values; The calibration plate is fixed on the stage of the measuring microscope, multiple images containing the calibration circle are collected by the microscope system, the initial image coordinates of the calibration circle are extracted by image processing software, and the extracted coordinates are converted into normalized coordinates; Based on the geometric structure of the calibration plate, the theoretical center coordinates of the calibration circle are calculated, and the difference between the actual detected center coordinates and the theoretical center coordinates is optimized by the least squares fitting method to obtain the accurate theoretical center coordinates; Detect radial distortion, eccentric distortion and thin prism distortion in the microscope system and calculate the corresponding distortion amount; The image is corrected for distortion using the calculated radial distortion, eccentric distortion and thin prism distortion; Based on the central perspective projection relationship, a perspective projection model is constructed and a perspective transformation matrix is ​​calculated, and the perspective error is corrected through the perspective transformation matrix; Analyze edge positioning deviation through edge detection algorithm, and correct edge detection error by adjusting detection parameters and algorithm; The accuracy of the calibration results is verified by using standard scale objects, and the verified calibration parameters are fixed into the ISP chip to realize the real-time calibration function of the measuring microscope.

2. The ISP chip-based measuring microscope calibration method according to claim 1, characterized in that: Fix the calibration plate on the stage of the measuring microscope, use the microscope system to collect multiple images containing the calibration circle, use image processing software to extract the initial image coordinates of the calibration circle, and convert the extracted coordinates into normalized coordinates. The specific steps include the following: The first step is to fix the calibration plate on the stage of the measuring microscope to ensure that the horizontal and vertical positions of the calibration plate are accurate to prevent image deviation caused by tilt or movement of the calibration plate. Step 2: Use the ISP chip to control the measuring microscope camera and collect multiple calibration plate images; The third step is to use the image processing system to extract the feature points in the calibration plate image and convert the extracted feature point coordinates into normalized coordinates; The calculation formula for normalized coordinates is as follows: Among them, (c x , c y ) is the optical center, that is, the coordinate of the intersection of the optical axis and the image plane, (x d ,y d ) is the normalized image coordinate, and (x, y) represents the image coordinate.

3. The ISP chip-based measuring microscope calibration method according to claim 1, characterized in that: The calculation formula of the radial distortion is as follows: Among them, δ xr , δ yr They represent the horizontal and vertical distortion caused by radial distortion, q is the radial distortion coefficient of the lens, O(x d ,y d ) 5 Represents about x d and d High-order small quantities of fifth order and above; (x d ,y d ) are the normalized image coordinates.

4. The ISP chip-based measuring microscope calibration method according to claim 3, characterized in that: The calculation formula of the eccentric distortion is as follows: In the formula, δ xp , δ yp They represent the horizontal and vertical distortion caused by eccentric distortion respectively; p1 and p2 represent the eccentric distortion coefficients, O(x d ,y d ) 4 Represents about x d and d Higher-order small quantities of fourth order and above.

5. The ISP chip-based measuring microscope calibration method according to claim 4, characterized in that: The calculation formula of the thin prism distortion is as follows: In the formula, δ xs , δ ys Respectively represent the horizontal and vertical distortion caused by thin prism distortion; s1 and s2 represent the thin prism distortion coefficient, O(x d ,y d ) 4 Represents about x d and d Higher-order small quantities of fourth order and above.

6. The ISP chip-based measuring microscope calibration method according to claim 5, characterized in that: The coordinates of the image are corrected by using the calculated radial distortion, eccentric distortion, and distortion in the horizontal and vertical directions caused by thin prism distortion to obtain the calculation formula for the coordinates after image correction as follows: Among them, x p and p represents the coordinates of the image after distortion correction, δ xr , δ yr They represent the horizontal and vertical distortion caused by radial distortion, δ xp , δ yp They represent the horizontal and vertical distortion caused by eccentricity distortion, δ xs , δ ys They represent the horizontal and vertical distortions caused by thin prism distortion.

7. The ISP chip-based measuring microscope calibration method according to claim 1, characterized in that: After converting the initial image coordinates of the calibration circle into normalized coordinates, the Harris corner detection algorithm can be used in the normalized coordinate image to detect the corner positions in the calibration plate image. The detected corner coordinates are refined using a sub-pixel precision algorithm to improve the accuracy of calibration circle detection.

8. The ISP chip-based measuring microscope calibration method according to claim 1, characterized in that: The calculation formula of the theoretical center coordinates is: in and They represent the theoretical center coordinates of the calibration circle. The calibration circle adopts a 3*3 equal spacing layout. A r It means [x 3 y 3 …x 1 y 1 ], i and j represent the number of rows and columns of the calibration circle, respectively, a and b represent the theoretical center coordinates of the calibration circle in the middle area of ​​the calibration plate image, and k represents the slope value determined by the center coordinates of the calibration circle with three rows and three columns according to the principle of least squares method.

9. The ISP chip-based measuring microscope calibration method according to claim 1, characterized in that: Based on the central perspective projection relationship, a perspective projection model is constructed and the perspective transformation matrix is ​​calculated. The perspective error is corrected by the perspective transformation matrix. The specific calculation formula is as follows: From the above formula, we can get: Among them, x q ,y q 、z q represents the coordinates of the image after perspective error correction, x, y, and z represent the coordinates of the image without perspective error correction, z represents the distance from the center of the microscope objective lens to the CCD, which is a constant, α represents the angle of rotation around the x-axis when the image is not corrected for perspective error, and β represents the angle of rotation around the y-axis when the image is not corrected for perspective error.

Images