Calibration method and device of high-precision imaging system and image stitching method, device and medium

Abstract

The application discloses a kind of high-precision imaging system's calibration and image splicing method, device and medium, the method includes the following steps: subpixel accuracy center coordinates are paired with known physical coordinates of calibration board, plane homography matrix is solved and X and Y direction resolution are calculated;Multi-direction motion sampling is collected calibration data, and the mechanical system comprehensive model containing multidimensional error component is established, and the model parameters of the mechanical system comprehensive model are solved;After rough calibration, construct multi-view field global optimization model, fuse calibration board multi-attitude cross observation data, create global re-projection error optimization problem, solve fine calibration parameters;Based on fine calibration parameter, construct the coordinate mapping relationship between view field pixel coordinates and global coordinate system, based on its reverse mapping, execute image resampling and pixel fusion;The application can realize subpixel level calibration and seamless splicing by rough-fine hierarchical calibration, accurate feature extraction and intelligent splicing technology.

Description

Technical Field

[0001] This invention relates to the field of industrial machine vision, and in particular to a calibration and image stitching method, apparatus and medium for a high-precision imaging system. Background Technology

[0002] Existing image stitching and camera calibration technologies can be mainly divided into the following four categories:

[0003] (1) Image feature-based stitching methods rely on feature point matching such as SIFT and SURF, which are prone to false matching due to texture repetition or feature loss, resulting in high computational complexity.

[0004] (2) The mechanical motion control method relies on high-precision stepper motors and grating rulers, which are costly and difficult to achieve sub-pixel accuracy, resulting in cumulative errors;

[0005] (3) The method of combining mechanical platform with image processing still faces the problem of coordinating mechanical and algorithmic aspects, and the operation difficulty and cost remain high;

[0006] (4) The method of combining Zhang Zhengyou's calibration method with splicing is complicated to operate, time-consuming, and depends on a stable shooting environment.

[0007] The aforementioned technologies all struggle to balance calibration accuracy, splicing quality, operational efficiency, and cost control, failing to meet the micron-level precision and high-efficiency production requirements of industrial automated optical inspection. Therefore, a systematic high-precision solution is urgently needed. Summary of the Invention

[0008] The purpose of this invention is to provide a calibration and image stitching method, apparatus and medium for a high-precision imaging system, thereby solving all or one of the aforementioned problems in the prior art.

[0009] To solve the above-mentioned technical problems, the specific technical solution of the present invention is as follows:

[0010] On one hand, the present invention provides a calibration and image stitching method for a high-precision imaging system, comprising the following steps:

[0011] Calibration board image preprocessing and feature extraction:

[0012] The original calibration board image is sequentially subjected to filtering and noise reduction, binarization and morphological optimization, as well as contour detection and screening. Then, sub-pixel level edge thinning and fitting are used to extract the sub-pixel precision center coordinates of the circular markers.

[0013] Camera resolution calibration:

[0014] Pair the subpixel accuracy center coordinates with the known physical coordinates of the calibration board, solve the planar homography matrix, and decompose the scale factor of the planar homography matrix; calculate the X and Y direction resolutions based on the scale factor and the physical spacing of feature points on the calibration board, and determine the final resolution parameters based on multi-position resolution data and reprojection error verification;

[0015] Coarse calibration:

[0016] Calibration data is collected through multi-directional motion sampling to establish a comprehensive mechanical system model containing multi-dimensional error components; the model parameters of the comprehensive mechanical system model are solved using a hierarchical optimization strategy, and the initial transformation relationship from the mechanical coordinate system to the visual coordinate system is obtained based on accuracy verification and uncertainty assessment.

[0017] Precision calibration:

[0018] After coarse calibration, a multi-field global optimization model is constructed, and multi-pose cross-observation data of the calibration board are fused to create a global reprojection error optimization problem. The objective function for minimizing the residuals of the global reprojection error optimization problem is defined, and the coordinate consistency constraint of the overlapping area is enhanced. The optimization problem is solved nonlinearly to obtain the fine calibration parameters.

[0019] Image stitching and error control:

[0020] A global coordinate system is established, and a coordinate mapping relationship between the field-of-view pixel coordinates and the global coordinate system is constructed based on the fine calibration parameters; image resampling and pixel fusion are performed based on the inverse mapping of the coordinate mapping relationship.

[0021] As an improved approach, the filtering and noise reduction includes: performing a convolution operation between a two-dimensional Gaussian kernel and the original calibration plate image;

[0022] The binarization and morphological optimization include: calculating the grayscale threshold of the filtered image using the Otsu algorithm; and sequentially performing erosion and dilation operations on the Otsu binarized image using circular or square structuring elements.

[0023] The contour detection and screening includes: using an edge tracking algorithm in conjunction with the RETR_EXTERNAL retrieval mode to extract the closed contour of the binarized and morphologically optimized image, and screening candidate contours based on the expected area range of the calibration plate markers, the contour perimeter, and the duty cycle.

[0024] As an improved approach, the subpixel-level edge refinement and fitting includes: refining the contour edges of the candidate contour using the gray-scale moment method, and substituting the refined edge points into the least-squares ellipse fitting model to output the center coordinates of the circular marker point.

[0025] As an improved solution, the X and Y direction resolutions include: X direction resolution and Y direction resolution;

[0026] The formula for calculating the resolution in the X direction is: ;

[0027] The formula for calculating the resolution in the Y direction is: ;

[0028] in, and The magnitudes of the first two columns of the homography matrix; The physical spacing between feature points on the calibration board.

[0029] As an improved approach, the multi-position resolution data is the average or median of the X and Y direction resolutions of calibration board images at several different positions and orientations.

[0030] As an improved solution, the multi-directional motion sampling further includes: collecting data along four directions: positive X-axis, negative X-axis, positive Y-axis, and negative Y-axis, and simultaneously recording PLC command coordinates, encoder feedback values, and motion direction indicators;

[0031] The multi-dimensional error components include: proportional error components, orthogonal error components, and nonlinear positioning error components;

[0032] The hierarchical optimization strategy includes: solving for linear transformation parameters, optimizing backlash parameters and orthogonal error angles, and optimizing nonlinear error mapping tables layer by layer; the hierarchical optimization strategy employs the Levenberg-Marquardt algorithm and the Huber loss function.

[0033] As an improved approach, the objective function for minimizing the residuals is:

[0034] ;

[0035] in, It is the field of view k for physical points Observed subpixel image coordinates; It is a complete camera projection model function that includes intrinsic parameters, distortion transformation, and extrinsic parameter transformation; , These are the rotation and translation parameters from the world coordinate system to the k-th field-of-view camera coordinate system; It is a weighting factor; These are lens distortion parameters; It's the camera's internal parameters; Physical world point located on the calibration board of attitude j ; , These belong to the aforementioned fine calibration parameters;

[0036] The enhanced coordinate consistency constraint for overlapping regions further includes:

[0037] Set a higher weight factor for overlapping points than for non-overlapping points.

[0038] As an improved approach, the step of constructing a coordinate mapping relationship between the field-of-view pixel coordinates and the global coordinate system based on the fine calibration parameters includes:

[0039] For each field of view k obtained by the fine calibration, the camera intrinsic parameters, lens distortion parameters and camera extrinsic parameters are used to calculate the forward projection function from the global three-dimensional coordinates to the pixel coordinates of the field of view image;

[0040] The forward projection function is: ;

[0041] The reverse mapping of the coordinate mapping relationship is: .

[0042] On the other hand, the present invention also provides a calibration and image stitching device for a high-precision imaging system, comprising:

[0043] The feature extraction module is used to: sequentially perform filtering and noise reduction, binarization and morphological optimization, and contour detection and screening operations on the original calibration board image, and then extract the sub-pixel precision center coordinates of the circular marker points through sub-pixel level edge thinning and fitting;

[0044] The camera resolution calibration module is used to: pair the sub-pixel accuracy center coordinates with the known physical coordinates of the calibration board, solve the plane homography matrix and decompose the scale factor of the plane homography matrix; calculate the X and Y direction resolutions based on the scale factor and the physical spacing of feature points on the calibration board; and determine the final resolution parameters based on multi-position resolution data and reprojection error verification.

[0045] The coarse calibration module is used to: collect calibration data through multi-directional motion sampling, establish a comprehensive mechanical system model containing multi-dimensional error components; solve the model parameters of the comprehensive mechanical system model using a hierarchical optimization strategy, and obtain the initial transformation relationship from the mechanical coordinate system to the visual coordinate system based on accuracy verification and uncertainty assessment;

[0046] The fine calibration module is used to: after coarse calibration, construct a multi-field global optimization model, fuse multi-pose cross-observation data of the calibration board, and create a global reprojection error optimization problem; define a residual minimization objective function for the global reprojection error optimization problem and enhance the coordinate consistency constraint of the overlapping area, perform nonlinear iterative solution to the optimization problem, and obtain the fine calibration parameters;

[0047] The image stitching and error control module is used to: establish a global coordinate system and construct a coordinate mapping relationship between the field-of-view pixel coordinates and the global coordinate system based on the fine calibration parameters; and perform image resampling and pixel fusion based on the reverse mapping of the coordinate mapping relationship.

[0048] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the calibration and image stitching method of the high-precision imaging system.

[0049] On the other hand, the present invention also provides a computer device, the computer device including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; wherein:

[0050] The memory is used to store computer programs;

[0051] The processor is used to execute the steps of the calibration and image stitching method of the high-precision imaging system by running the program stored in the memory.

[0052] The beneficial effects of the technical solution of this invention are:

[0053] 1. The calibration and image stitching method of the high-precision imaging system described in this invention can achieve sub-pixel-level calibration and seamless stitching through coarse and fine hierarchical calibration, precise feature extraction and intelligent stitching technology, which significantly improves calibration accuracy and stitching quality; simplifies the operation process, improves efficiency, and reduces hardware cost dependence; enhances the robustness and adaptability of the system in complex environments, makes up for the defects of the prior art, and has high application value.

[0054] 2. The calibration and image stitching device for the high-precision imaging system described in this invention can realize the calibration and image stitching method for the high-precision imaging system described in this invention through the cooperation of the device modules.

[0055] 3. The computer-readable storage medium of the present invention can cooperate with the guidance device module to realize the calibration and image stitching method of the high-precision imaging system of the present invention. The computer-readable storage medium of the present invention also effectively improves the operability of the calibration and image stitching method of the high-precision imaging system. Attached Figure Description

[0056] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating the calibration and image stitching method of the high-precision imaging system described in Embodiment 1 of the present invention;

[0058] Figure 2 This is a schematic diagram of the logic flow of the calibration and image stitching method of the high-precision imaging system described in Embodiment 1 of the present invention;

[0059] Figure 3 This is a schematic diagram of the architecture of the calibration and image stitching device for the high-precision imaging system described in Embodiment 2 of the present invention. Detailed Implementation

[0060] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0061] In the description of this invention, it should be noted that the embodiments described in this invention are only some embodiments of this invention, not all embodiments; based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0062] The terms "first," "second," etc., used in this specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0063] In the description of this invention, it should be noted that the image feature-based stitching method in the background art is as follows: by extracting feature points in the image, such as SIFT, SURF, etc., images under different FOVs (fields of view) are registered and stitched together; the advantage of this type of method is that it does not rely on mechanical precision, but when facing images with periodic repeating textures or few feature points, it is easy to produce false matching of feature points, resulting in stitching errors and failing to meet the requirements of micron-level precision; in addition, the traditional stitching method based on direct comparison of pixel points has a large computational load in high-resolution images and is difficult to apply in real time.

[0064] In the description of this invention, it should be noted that the stitching method based on mechanical motion control in the background art is as follows: using a precision mechanical system, such as a stepper motor and a grating ruler, the movement of the image acquisition device is controlled to obtain images with equal spacing, and then the images are stitched together sequentially. The stitching accuracy of this method depends on the accuracy of the mechanical motion platform, so the cost is high and it is difficult to achieve sub-pixel level stitching accuracy. In addition, due to the existence of machining and installation errors, this method has limited performance in scenarios requiring micron-level accuracy, especially when the image resolution or imaging area is large.

[0065] In the description of this invention, it should be noted that the stitching method combining mechanical platform and image processing in the background art is as follows: for example, using a high-precision grating ruler to perform feedback control on the movement of the acquisition device to reduce the influence of mechanical errors, and then combining image features for stitching correction; such methods can achieve high stitching accuracy in one dimension, but for stitching multiple images in two dimensions, there is still a defect that the accumulation of errors is not considered, which makes it difficult to guarantee the overall accuracy of stitching in high-precision detection.

[0066] In the description of this invention, it should be noted that the combination of Zhang Zhengyou's calibration method and image stitching in the background art is as follows: the intrinsic and extrinsic parameters of the camera are estimated by shooting the calibration board at different angles; however, the existing calibration method is mainly for the calibration of the camera imaging system and has not fully integrated with image stitching technology. Especially in industrial scenarios with high precision requirements, it cannot solve the problem of inaccurate measurement caused by inaccurate equipment installation or image stitching errors.

[0067] Example 1: This example provides a calibration and image stitching method for a high-precision imaging system, such as... Figure 1 and Figure 2 As shown, it includes the following steps:

[0068] S100, Calibration board image preprocessing and feature extraction, including:

[0069] In this step, the center coordinates of the circular marker points are extracted from the original calibration board image with high precision and robustness through a series of image processing operations. The specific steps are as follows:

[0070] S101. Image Filtering and Noise Reduction: Gaussian filtering and smoothing are performed on the original calibration board image. Specifically, a two-dimensional Gaussian kernel with a standard deviation σ=1.5 is selected (this standard deviation can be adjusted according to the actual noise situation). The two-dimensional Gaussian kernel is convolved with the original calibration board image. This operation suppresses high-frequency sensor noise (including salt-and-pepper noise and Gaussian noise) and small-scale texture interference in the original image, while preserving image edge information and improving the image signal-to-noise ratio, providing basic image data for subsequent binarization and edge detection.

[0071] S102, Image Binarization and Morphological Optimization:

[0072] S1021 Adaptive Threshold Segmentation: Apply the Otsu algorithm to the image filtered in step S101. The algorithm automatically calculates the optimal grayscale threshold of the image and converts the filtered image into a black and white binary image based on the optimal grayscale threshold, thereby separating the foreground (circular markers) from the background (calibration board substrate) to cope with scenes with uneven global illumination.

[0073] S1022, Morphological Opening Operation: First, select a 3×3 circular or square structuring element and perform an erosion operation on the binary image obtained in step S1021. The erosion breaks the small connections in the image and eliminates speckle noise. Then, using a structuring element of the same size and type, perform a dilation operation on the eroded image to restore the circular markers to approximately the original size and shape, without restoring the noise removed by the erosion. Finally, a binary image with independent and complete contours is obtained.

[0074] S103. Contour Detection and Preliminary Screening:

[0075] S1031. Contour Extraction: The edge tracking algorithm in the connected component analysis algorithm (such as the findContours algorithm in conjunction with the RETR_EXTERNAL retrieval mode) is used to process the binary image obtained after morphological processing in step S1022 and extract all closed contours in the image.

[0076] S1032. Contour Filtering: Based on the prior knowledge of the calibration board marker points (including preset parameters such as expected area range, contour perimeter, and duty cycle), all closed contours extracted in step S1031 are filtered one by one. Invalid contours with areas exceeding the expected range (too large may be stains or reflective areas, too small may be noise points) are removed, and candidate marker point contours that conform to preset geometric features are retained.

[0077] S104, Sub-pixel level center coordinate extraction:

[0078] S1041, Subpixel refinement of edge points: For each candidate contour obtained in step S1032, the gray-level gradient information of the image is used to perform edge localization processing on the contour edge using the gray-level moment method, thereby improving the edge localization accuracy to the subpixel level of 0.1 pixels.

[0079] S1042, Ellipse / Circle Fitting: Substitute the subpixel precision edge point set obtained in step S1041 into the least squares ellipse fitting model to solve for the ellipse parameters (including center coordinates (cx, cy), major and minor axes (a, b), and rotation angle θ). The model aims to minimize the sum of the algebraic distances from all edge points to the ellipse. If the image is a positive viewpoint or distortion correction has been completed, the model is simplified to circle fitting to solve for the circle parameters.

[0080] S1043, Center Coordinate Output: The center (cx, cy) of the ellipse or circle obtained by fitting in step S1041 is used as the final feature point coordinates of the corresponding marker point. These coordinates are floating-point numbers with sub-pixel precision.

[0081] S200, camera resolution calibration, including:

[0082] In this step, by establishing a spatial scale reference between the pixel coordinate system and the world coordinate system, the actual physical size corresponding to a pixel in the image plane is determined. The specific steps are as follows:

[0083] S201. Calculate the homography matrix and scale factor based on feature points:

[0084] S2011, Feature point pairing: Pair all the sub-pixel image coordinates of the circular markers obtained in step one ( , ), and the corresponding physical coordinates in the predefined calibration plate physical coordinate system (with the upper left corner of the calibration plate as the origin, and the unit being millimeters). , ,0) for precise pairing, where the calibration plate plane is a flat surface and the physical Z coordinates of all marker points are set to 0.

[0085] S2012, Solving the Planar Homography Matrix: Using the RANSAC robust estimation algorithm, the feature point data after pairing in step S2011 is processed to solve for the 3×3 homography matrix H. This matrix describes the projection mapping relationship from the calibration plate's physical plane to the image plane, satisfying the formula... , where s is the scaling factor.

[0086] S2013, Decomposing the homography matrix: Decompose the homography matrix H obtained in step S2011 into a combination of the camera intrinsic parameter matrix K, the rotation matrix R, and the translation vector t (i.e., ,in and (These are the first two columns of the rotation matrix R). Given K (calculated through distortion correction), extract the scale information; or directly calculate the magnitude of the first two columns of the homography matrix. and These two represent the pixel lengths of the X-axis and Y-axis unit vectors in the physical coordinate system after they are mapped to the image.

[0087] S202. Calculate the resolution in the X and Y directions:

[0088] S2021, Scale Factor Calculation: Obtaining the known precise physical distance between feature points on the calibration board. (Unit: mm), obtained in conjunction with step S2013 and Calculate the camera's resolution in the X and Y directions separately. The formula is: X-direction resolution (Unit: mm / pixel), Y-axis resolution (Unit: mm / pixel), where ||h1|| and ||h2|| are statistical values ​​(such as the mean) calculated using all successfully paired feature points.

[0089] S2022, Multi-position acquisition and averaging: Acquire images of the calibration board at 10-20 different positions and orientations within the camera's field of view. Repeat steps S201 and S2021 for each image to calculate 10-20 sets of Sx and Sy values. Calculate the average or median of all Sx values ​​as the final resolution parameter in the X direction of the camera, and calculate the average or median of all Sy values ​​as the final resolution parameter in the Y direction of the camera. Simultaneously, calculate the standard deviation of Sx and Sy to evaluate the repeatability accuracy of the calibration results.

[0090] S203. Accuracy Verification and Evaluation: In a preferred embodiment, using the final resolution parameters Sx and Sy obtained in step S202, the theoretical pixel distance of feature points with known physical distances on the image is calculated. The theoretical pixel distance is compared with the actual pixel distance detected from the image, and the reprojection error is calculated. The accuracy of this resolution calibration is quantified by the average value and distribution of the error.

[0091] S300, coarse calibration, including:

[0092] In this step, through multi-directional motion sampling, systematic error modeling, and hierarchical optimization, an initial transformation relationship from the mechanical coordinate system to the visual coordinate system is constructed to compensate for mechanical installation deviations and positioning errors, thus establishing an accurate mechanical positioning benchmark for subsequent fine calibration. The specific steps are as follows:

[0093] S301, Multi-directional sampling and data acquisition:

[0094] S3011, Calibration plate placement: Place the high-precision planar calibration plate on the working plane within the camera's field of view, ensuring that the calibration plate is approximately parallel to the working plane.

[0095] S3012, Multi-directional movement and data acquisition: Control the motion platform to move the calibration board along a predetermined path, so that the calibration board covers multiple different positions within the entire field of view of the camera. Each position moves along four directions: positive X-axis, negative X-axis, positive Y-axis, and negative Y-axis. When the calibration board moves to the target position, perform the following operations simultaneously: record the current PLC command coordinates and encoder feedback values, trigger the camera to acquire images of the calibration board, and mark the current movement direction. For each acquired image, perform the feature extraction operation in step S100 to obtain the PLC position and the known fixed physical coordinates of the calibration board corresponding to the image. , The pairing relationship of 0) ultimately forms a complete set of data pairs {PLC instruction position, motion direction, image feature point set}. This method overcomes the problems of single position calibration being sensitive to noise, unable to cover the entire field of view, and difficult to separate the mechanical transmission system's own errors from the calibration parameters.

[0096] S302. Mechanical Error Characteristics Analysis and Modeling:

[0097] S3021, Backlash Calculation: From the multi-directional motion data acquired in step S3012, separate the forward and reverse motion trajectories; for images that reach the same physical position but have different motion directions, calculate the precise position of the calibration plate separately using the formula. Calculate the X-axis backlash, and then calculate the Y-axis backlash using the same method.

[0098] S3022. Establishment of a comprehensive mechanical system model:

[0099] Establish a proportional error component model: , ;in , This represents the actual transmission ratio of each shaft;

[0100] Calculate the orthogonal error components: By analyzing the oblique motion data, the actual included angle of the mechanical axis and the deviation from the theoretical 90° are obtained;

[0101] Establish a nonlinear positioning error model: Set a test point every 10mm within the travel range of the motion axis and construct a position-related error mapping table;

[0102] Establish coordinate transformation relationships: integrate rotation angle, translation offset, and anisotropic scaling parameters to form a 4×4 homogeneous transformation matrix.

[0103] S303, Hierarchical Optimization and Parameter Solving:

[0104] S3031, First-level optimization: Solve only the linear transformation parameters (including rotation matrix, scaling factor, and translation vector), and set the nonlinear error components to zero; use the data from all sampling points to establish the objective function (minimize) using the least squares method. The linear transformation parameters are obtained by solving the problem.

[0105] S3032, Second layer optimization: Based on the results of the first layer optimization, the backlash parameter and orthogonal error angle are optimized; the forward and reverse motion data are processed separately, and the backlash value of each axis is accurately solved through comparative analysis. At the same time, the orthogonal error angle is optimized to correct the non-perpendicularity of the mechanical coordinate system.

[0106] S3033, Third Layer Optimization: Based on the results of the second layer optimization, the nonlinear error mapping table is further optimized; a smooth nonlinear error compensation function is established using cubic spline interpolation; by optimizing the position of the interpolation control points, the model accurately compensates for position-related positioning deviations; throughout the optimization process, the Huber loss function is used to enhance the robustness of the algorithm and reduce the impact of outlier data points; the Levenberg-Marquardt algorithm is selected for optimization, with dual convergence conditions set: parameter change less than 0.001 or objective function decrease less than 1 × 10⁻⁶. -6 After each iteration, the root mean square value of the reprojection error is calculated to monitor the convergence.

[0107] S304. Accuracy Verification and Uncertainty Assessment:

[0108] S3041. Accuracy Verification: Acquire new calibration board images at verification locations that were not calibrated. Use the coarse calibration parameters obtained in step S300 to predict the calibration board position. Calculate the deviation between the predicted position and the actual position extracted from the image. The maximum deviation in the entire field of view should be less than 0.01 mm, and the root mean square deviation should be less than 0.005 mm.

[0109] S3042 Uncertainty Assessment: Calculate the covariance matrix of the parameter estimates using the Hessian matrix of the optimization problem, and take the square root of the diagonal elements of the covariance matrix as the uncertainty of each parameter; output the mechanical-vision transformation matrix, the scaling factor of each axis, the backlash value, the orthogonal error angle, the nonlinear error compensation table, and the uncertainty estimate of each parameter.

[0110] S400, fine calibration, including:

[0111] In this step, based on the initial parameters of coarse calibration, the calibration accuracy and coordinate consistency of overlapping areas of adjacent fields of view are improved through multi-field data fusion and global optimization. The specific steps are as follows:

[0112] S401. Constructing a global optimization model and fusing multi-view data:

[0113] S4011. Global Optimization Model Construction: Assume the system contains M fields of view, and each field of view k corresponds to the camera intrinsic parameters. External parameters , and lens distortion parameters (Including radial distortion coefficients k1, k2, k3 and tangential distortion coefficients p1, p2).

[0114] S4012, Multi-pose data acquisition: The high-precision planar calibration board is moved in multiple different poses j in all fields of view to ensure that the feature points of the calibration board in each pose can be clearly captured by at least two fields of view, forming a cross-observation constraint.

[0115] S4013, Data Fusion: Collect all acquired images to form an observation dataset, and fuse the data from the same physical world point. (Located on the calibration plate at attitude j) and the pixel coordinates observed from multiple fields of view k (u ijk ,v ijk This is related to the global reprojection error optimization problem.

[0116] S402. Define the optimization objective function: Establish an objective function with residual minimization as its core:

[0117] ;

[0118] in, It is the field of view k for physical points Observed subpixel image coordinates It is a complete camera projection model function that includes intrinsic parameters, distortion transformation, and extrinsic parameter transformation. , It is the rotation and translation from the world coordinate system to the k-th field-of-view camera coordinate system. It is a weighting factor (the default value is 1, which can be adjusted based on the feature point detection confidence or whether it is an overlapping area point).

[0119] S403, Specific optimizations for splicing boundaries:

[0120] S4031. Feature point identification in the boundary region: In the observation dataset of step S4013, identify the feature points located in the overlapping region of any two fields of view (i.e., feature points observed simultaneously by two or more fields of view).

[0121] S4032. Introduce consistency constraints:

[0122] Direct method: In the objective function of step 2, assign higher weights to feature points in the overlapping region. (e.g., non-overlapping points have a weight of 1, and overlapping points have a weight of 2 or higher), which forces the reprojection error of feature points in the overlapping region to be minimized in all relevant fields of view.

[0123] Indirect method / post-processing: After completing a global optimization, calculate the coordinate difference between each overlapping point and the world coordinates in different fields of view. Add this difference as the "inter-field registration error" to the objective function and perform iterative optimization again until the error is lower than the sub-pixel threshold.

[0124] S404. Nonlinear Iterative Solution and Accuracy Verification:

[0125] S4041. Nonlinear Iterative Solution: The sparse Levenberg-Marquardt algorithm is used to solve large-scale nonlinear least squares problems, leveraging the sparsity of the Jacobian matrix to improve solution efficiency; the intrinsic parameters of all fields of view are iteratively adjusted. Distortion parameters External reference , And the external parameters of each attitude of the calibration board [ , This continues until the objective function meets the convergence condition.

[0126] S4042, Accuracy Verification:

[0127] Calculate the root mean square error of the reprojection of all feature points as the overall calibration accuracy index;

[0128] Feature points in the overlapping area are selected and back-projected from different fields of view to the world coordinate system using optimized parameters. The 3D coordinate difference of the same point under different fields of view is calculated, and the mean and maximum value of the difference are used as the core criteria for the geometric consistency accuracy of multi-field stitching.

[0129] S500, Image Stitching and Error Control:

[0130] In this step, the transformation relationship obtained from precise calibration is used to fuse multiple independent field-of-view images into a large-format image with global geometric consistency. Error control ensures stitching accuracy. The specific steps are as follows:

[0131] S501. Establish a global coordinate system and coordinate mapping:

[0132] S5011. Define the global coordinate system: Select the camera coordinate system or virtual world coordinate system of a certain field of view (such as the central field of view) as the unified global coordinate system.

[0133] S5012, Calculate the forward mapping relationship: For each field of view k obtained from the fine calibration, the camera parameters (intrinsic parameters) ,distortion External parameters , ]), calculate the forward projection function from global 3D coordinates to the pixel coordinates of the image in this field of view. If the scene is planar (with a constant Z-coordinate, such as the plane where the calibration plate is located), then the function degenerates into a homography matrix. It directly maps global 2D coordinates to pixel coordinates of the field of view k.

[0134] S5013. Calculate the inverse mapping relationship: Solve the inverse transformation of the forward mapping in step S5012 to obtain the inverse mapping function. This function defines the precise location of each input pixel in the final output large image.

[0135] S502. Image resampling and fusion based on reverse mapping:

[0136] S5021. Create a blank canvas: Based on the coverage of all view areas in the global coordinate system, create a blank output image (canvas) of a sufficiently large size.

[0137] S5022, Traverse the output image pixels: For each target pixel of the output image Using the inverse mapping function calculated in step S5013, the sub-pixel source coordinates of each point in each input field of view are calculated one by one. .

[0138] S5023, Intelligent Interpolation Sampling: For sub-pixel level source coordinates The bicubic interpolation algorithm is used to sample grayscale / color values ​​from the image of the input field of view k. Specifically, 16 pixels in a 4×4 neighborhood around the source point are selected, and the weighting coefficient of each pixel is calculated based on the distance using a cubic spline function. The grayscale / color values ​​of the 16 pixels are then weighted and averaged to obtain the sampling result.

[0139] S5024, Pixel Blending: If the output pixels Within the overlapping region of multiple input fields of view, a weighted fusion strategy is employed:

[0140] Simple average: Take the average of the interpolation results of all valid fields of view at this location;

[0141] Distance weight: Based on the pixel's position in its respective field of view (such as its distance from the center of the field of view) or feathering weight, the contribution values ​​of multiple fields of view are weighted and averaged to achieve a smooth transition in the overlapping area.

[0142] S503, Systematic Error Analysis and Compensation:

[0143] S5031, Reprojection Error Analysis: Select calibration board points or additional verification patterns as physical feature points, calculate the pixel positions of the same physical feature points from different fields of view on the stitched image, calculate the Euclidean distance (i.e., position deviation) of these positions, statistically analyze the deviations of all verification points, calculate the root mean square error and the maximum error, and generate a quantitative stitching accuracy report.

[0144] S5032, Error Source Tracing and Compensation:

[0145] Residual model learning: If the systematic error exhibits regularity (e.g., the deviation at the edge of the field of view is greater than that at the center), the positional deviation measured in step 3.1 is modeled as a function of the point's position in the image (e.g., a low-order polynomial surface), thus obtaining a residual error model describing the actual observation stitching error. ;

[0146] Online compensation: In the reverse mapping calculation of step S5022, the residual error model is introduced as a compensation term, and the new mapping relationship is as follows: Known systematic errors are pre-compensated during the image generation stage.

[0147] S5033, Boundary Treatment and Dynamic Adjustment:

[0148] Effective region mask: Generate an effective pixel region mask for each field of view, and only map and fuse regions with good optical quality and reliable distortion correction, excluding regions with blurred boundaries or excessive distortion;

[0149] Seam optimization: In the overlapping area, an image stitching algorithm based on optimal seam search is used to fuse along the path with the minimum gradient and the minimum color difference in the image content, thereby minimizing the visible seams of the stitching boundary.

[0150] It should be noted that the above examples are merely for explaining the present invention and should not be construed as limiting the scope of protection of the present invention.

[0151] Example 2: This example, based on the same inventive concept as the calibration and image stitching method for a high-precision imaging system described in Example 1, provides a calibration and image stitching device for a high-precision imaging system, such as... Figure 3 As shown, it includes:

[0152] The feature extraction module is used to: sequentially perform filtering and noise reduction, binarization and morphological optimization, and contour detection and screening operations on the original calibration board image, and then extract the sub-pixel precision center coordinates of the circular marker points through sub-pixel level edge thinning and fitting;

[0153] The camera resolution calibration module is used to: pair the sub-pixel accuracy center coordinates with the known physical coordinates of the calibration board, solve the plane homography matrix and decompose the scale factor of the plane homography matrix; calculate the X and Y direction resolutions based on the scale factor and the physical spacing of feature points on the calibration board; and determine the final resolution parameters based on multi-position resolution data and reprojection error verification.

[0154] The coarse calibration module is used to: collect calibration data through multi-directional motion sampling, establish a comprehensive mechanical system model containing multi-dimensional error components; solve the model parameters of the comprehensive mechanical system model using a hierarchical optimization strategy, and obtain the initial transformation relationship from the mechanical coordinate system to the visual coordinate system based on accuracy verification and uncertainty assessment;

[0155] The fine calibration module is used to: after coarse calibration, construct a multi-field global optimization model, fuse multi-pose cross-observation data of the calibration board, and create a global reprojection error optimization problem; define a residual minimization objective function for the global reprojection error optimization problem and enhance the coordinate consistency constraint of the overlapping area, perform nonlinear iterative solution to the optimization problem, and obtain the fine calibration parameters;

[0156] The image stitching and error control module is used to: establish a global coordinate system and construct a coordinate mapping relationship between the field-of-view pixel coordinates and the global coordinate system based on the fine calibration parameters; and perform image resampling and pixel fusion based on the reverse mapping of the coordinate mapping relationship.

[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0158] Example 3: This example provides a computer-readable storage medium, including:

[0159] The storage medium is used to store computer software instructions used to implement the calibration and image stitching method of the high-precision imaging system described in Embodiment 1. It includes a program for executing the calibration and image stitching method of the high-precision imaging system. Specifically, the executable program can be built into the calibration and image stitching device of the high-precision imaging system described in Embodiment 2. In this way, the calibration and image stitching device of the high-precision imaging system can implement the calibration and image stitching method of the high-precision imaging system described in Embodiment 1 by executing the built-in executable program.

[0160] Furthermore, the computer-readable storage medium in this embodiment can be any combination of one or more readable storage media, wherein the readable storage medium includes an electrical, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.

[0161] Unlike existing technologies, the calibration and image stitching method, apparatus and medium of the high-precision imaging system proposed in this application can achieve sub-pixel level calibration and seamless stitching through coarse and fine graded calibration, precise feature extraction and intelligent stitching technology, which significantly improves calibration accuracy and stitching quality; simplifies operation process, improves efficiency, reduces hardware cost dependence; and enhances the robustness and adaptability of the system in complex environments.

[0162] It should be understood that in the various embodiments of this document, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0163] It should also be understood that, in the embodiments herein, the term "and / or" is merely a description of the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following associated objects have an "or" relationship.

[0164] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0165] In the embodiments provided herein, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.

[0166] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments described herein, depending on actual needs.

[0167] Furthermore, the functional units in the various embodiments of this document can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0168] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this paper, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this paper. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0169] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A calibration and image stitching method for a high-precision imaging system, characterized in that, Includes the following steps: Calibration board image preprocessing and feature extraction: The original calibration board image is sequentially subjected to filtering and noise reduction, binarization and morphological optimization, as well as contour detection and screening. Then, sub-pixel level edge thinning and fitting are used to extract the sub-pixel precision center coordinates of the circular markers. Camera resolution calibration: Pair the subpixel accuracy center coordinates with the known physical coordinates of the calibration board, solve the planar homography matrix, and decompose the scale factor of the planar homography matrix; calculate the X and Y direction resolutions based on the scale factor and the physical spacing of feature points on the calibration board, and determine the final resolution parameters based on multi-position resolution data and reprojection error verification; Coarse calibration: Calibration data is collected through multi-directional motion sampling to establish a comprehensive mechanical system model containing multi-dimensional error components; the model parameters of the comprehensive mechanical system model are solved using a hierarchical optimization strategy, and the initial transformation relationship from the mechanical coordinate system to the visual coordinate system is obtained based on accuracy verification and uncertainty assessment. Precision calibration: After coarse calibration, a multi-field global optimization model is constructed, and multi-pose cross-observation data of the calibration board are fused to create a global reprojection error optimization problem. The objective function for minimizing the residuals of the global reprojection error optimization problem is defined, and the coordinate consistency constraint of the overlapping area is enhanced. The optimization problem is solved nonlinearly to obtain the fine calibration parameters. Image stitching and error control: A global coordinate system is established, and a coordinate mapping relationship between the field-of-view pixel coordinates and the global coordinate system is constructed based on the fine calibration parameters; image resampling and pixel fusion are performed based on the inverse mapping of the coordinate mapping relationship.

2. The calibration and image stitching method for a high-precision imaging system according to claim 1, characterized in that: The filtering and noise reduction includes: performing a convolution operation between a two-dimensional Gaussian kernel and the original calibration plate image; The binarization and morphological optimization include: calculating the grayscale threshold of the filtered image using the Otsu algorithm; and sequentially performing erosion and dilation operations on the Otsu binarized image using circular or square structuring elements. The contour detection and screening includes: using an edge tracking algorithm in conjunction with the RETR_EXTERNAL retrieval mode to extract the closed contour of the binarized and morphologically optimized image, and screening candidate contours based on the expected area range of the calibration plate markers, the contour perimeter, and the duty cycle.

3. The calibration and image stitching method for a high-precision imaging system according to claim 2, characterized in that: The subpixel-level edge refinement and fitting includes: refining the contour edges of the candidate contour using the gray-scale moment method, and substituting the refined edge points into the least squares ellipse fitting model to output the center coordinates of the circular marker point.

4. The calibration and image stitching method for a high-precision imaging system according to claim 1, characterized in that: The X and Y direction resolutions include: X direction resolution and Y direction resolution; The formula for calculating the resolution in the X direction is: ; The formula for calculating the resolution in the Y direction is: ; in, and The magnitudes of the first two columns of the homography matrix; The physical spacing between feature points on the calibration board.

5. The calibration and image stitching method for a high-precision imaging system according to claim 1, characterized in that: The multi-position resolution data refers to the average or median resolution in the X and Y directions of several calibration board images at different positions and orientations.

6. The calibration and image stitching method for a high-precision imaging system according to claim 1, characterized in that: The multi-directional motion sampling further includes: collecting data along four directions: positive X-axis, negative X-axis, positive Y-axis, and negative Y-axis, and simultaneously recording PLC command coordinates, encoder feedback values, and motion direction indicators; The multi-dimensional error components include: proportional error components, orthogonal error components, and nonlinear positioning error components; The hierarchical optimization strategy includes: solving for linear transformation parameters, optimizing backlash parameters and orthogonal error angles, and optimizing nonlinear error mapping tables layer by layer; the hierarchical optimization strategy employs the Levenberg-Marquardt algorithm and the Huber loss function.

7. The calibration and image stitching method for a high-precision imaging system according to claim 1, characterized in that: The objective function for minimizing the residual is: ； in, It is the field of view k for physical points Observed subpixel image coordinates; It is a complete camera projection model function that includes intrinsic parameters, distortion transformation, and extrinsic parameter transformation; , These are the rotation and translation parameters from the world coordinate system to the k-th field-of-view camera coordinate system; It is a weighting factor; These are lens distortion parameters; It's the camera's internal parameters; Physical world point located on the calibration board of attitude j ; , These belong to the aforementioned fine calibration parameters; The enhanced coordinate consistency constraint for overlapping regions further includes: Set a higher weight factor for overlapping points than for non-overlapping points.

8. The calibration and image stitching method for a high-precision imaging system according to claim 1, characterized in that: The step of constructing the coordinate mapping relationship between the field-of-view pixel coordinates and the global coordinate system based on the fine calibration parameters includes: For each field of view k obtained by the fine calibration, the camera intrinsic parameters, lens distortion parameters and camera extrinsic parameters are used to calculate the forward projection function from the global three-dimensional coordinates to the pixel coordinates of the field of view image; The forward projection function is: ; The reverse mapping of the coordinate mapping relationship is: .

9. A calibration and image stitching device for a high-precision imaging system, characterized in that, include: The feature extraction module is used to: sequentially perform filtering and noise reduction, binarization and morphological optimization, and contour detection and screening operations on the original calibration board image, and then extract the sub-pixel precision center coordinates of the circular marker points through sub-pixel level edge thinning and fitting; The camera resolution calibration module is used to: pair the sub-pixel accuracy center coordinates with the known physical coordinates of the calibration board, solve the plane homography matrix and decompose the scale factor of the plane homography matrix; calculate the X and Y direction resolutions based on the scale factor and the physical spacing of feature points on the calibration board; and determine the final resolution parameters based on multi-position resolution data and reprojection error verification. The coarse calibration module is used to: collect calibration data through multi-directional motion sampling, establish a comprehensive mechanical system model containing multi-dimensional error components; solve the model parameters of the comprehensive mechanical system model using a hierarchical optimization strategy, and obtain the initial transformation relationship from the mechanical coordinate system to the visual coordinate system based on accuracy verification and uncertainty assessment; The fine calibration module is used to: after coarse calibration, construct a multi-field global optimization model, fuse multi-pose cross-observation data of the calibration board, and create a global reprojection error optimization problem; define a residual minimization objective function for the global reprojection error optimization problem and enhance the coordinate consistency constraint of the overlapping area, perform nonlinear iterative solution to the optimization problem, and obtain the fine calibration parameters; The image stitching and error control module is used to: establish a global coordinate system and construct a coordinate mapping relationship between the field-of-view pixel coordinates and the global coordinate system based on the fine calibration parameters; and perform image resampling and pixel fusion based on the reverse mapping of the coordinate mapping relationship.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the calibration and image stitching method of the high-precision imaging system according to any one of claims 1 to 8.

Images