Calibration method and device for telecentric structured light system based on geometric center of checkerboard
By combining the Sobel operator and Hough transform with template matching and the gray-scale centroid method, high-precision extraction of the geometric center of the checkerboard pattern in the telecentric structured light system was achieved, solving the problem of insufficient calibration accuracy and improving the robustness and adaptability of the system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2026-02-10
- Publication Date
- 2026-06-02
Smart Images
Figure CN122134823A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and optical 3D measurement technology, specifically relating to a calibration method and apparatus for a telecentric structured light system based on the geometric center of a checkerboard grid. Background Technology
[0002] With the rapid development of intelligent manufacturing, reverse engineering, and machine vision, non-contact 3D shape measurement technology has been widely applied. Among them, structured light fringe projection technology has become one of the most mainstream 3D measurement methods due to its advantages such as high precision, high resolution, full-field measurement, and high speed. This system typically consists of a camera and a projector. Its basic principle is as follows: the projector projects coded structured light fringes (such as sinusoidal fringes) onto the object being measured; the deformed fringes, modulated by the object's surface height, are captured by the camera; absolute phase information is obtained through phase-shifting algorithms and phase unwrapping techniques; and finally, the phase information is converted into 3D spatial coordinates using triangulation principles.
[0003] In this process, system calibration is a key step in determining measurement accuracy. Its purpose is to accurately obtain the internal parameters of the camera and projector (such as focal length, principal point coordinates, distortion coefficients) as well as the external parameters between them (rotation matrix and translation vector), thereby establishing a mathematical mapping relationship between two-dimensional image pixels, phase values and three-dimensional world coordinates.
[0004] In existing technologies, the calibration of telecentric structured light systems mainly relies on the "reverse camera" model. The fundamental theory of this method was established by Song Zhang et al. in their 2006 paper "Novel method for structured light system calibration" published in *Optical Engineering*. Its core lies in constructing the pixel correspondence between the camera and the projector through phase-coordinate mapping. This scheme first uses the camera to acquire checkerboard images in different poses. Typically, the Harris algorithm is used to calculate the gray-level gradient structure tensor, and non-maximum suppression is applied to obtain the coarse coordinates of the pixel-level corner points. Then, a sub-pixel refinement algorithm based on local windows is employed: for example, using the gradient orthogonal iteration method, based on the gradient orthogonality assumption, iteratively approximating the true position by minimizing the error equation; or using a quadratic surface fitting method, performing polynomial fitting on the corner point neighborhood to analytically obtain the coordinates of the extreme points, which are then used as the final observation data of the camera. Subsequently, the absolute phase map is calculated by combining the synchronously projected phase-shifted fringes. The phase values at the corner points are obtained through interpolation, and the corresponding projector pixel coordinates are calculated in reverse, thereby establishing the correspondence between the camera and the projector.
[0005] However, in practical applications of this model, the feature extraction method based on corner points has inherent physical limitations: on the one hand, due to the limitation of optical imaging depth of field, defocusing will cause the high-frequency features of corner points to become blurred and degraded; on the other hand, there is a sharp reflectivity step in the black-and-white boundary area where the corner point is located, and the edge phase noise introduced therein will cause the inverse calculation of the projector pixel coordinates to be deviated, thus affecting the overall calibration accuracy.
[0006] To address the instability of the aforementioned corner features, existing research has begun exploring alternative solutions based on "center points." For example, Chinese authorized patent CN115359123B (Invention title: A projection light path calibration method based on checkerboard center point extraction) proposes a projection light path calibration method based on checkerboard center point extraction, aiming to reduce computational complexity and avoid the influence of perspective transformation. However, the acquisition of its center point still relies on the prior extraction and calculation of the checkerboard corner point coordinates, and does not fundamentally break away from the dependence on clear corner points. Similarly, Chinese published patent application CN112037284A (Invention title: Checkerboard grid center point detection method, camera distortion calibration method and system) points out that light source overflow can cause the gray value of corner points to fluctuate within a certain range, and proposes to determine the center point through image cropping and the intersection of axisymmetric lines for distortion calibration.
[0007] While the aforementioned existing technologies have verified the effectiveness of "using the center point to replace the corner point" to some extent, most of them still regard corner point detection as a preliminary step or are limited to the distortion correction stage. They have not yet formed a systematic calibration scheme that can simultaneously overcome telecentric defocus blur and edge phase noise.
[0008] Specifically, existing technologies still have the following drawbacks: First, defocus blurring causes gradient-based corner feature extraction to fail. Current technologies generally rely on algorithms like Harris to detect checkerboard corners, which is essentially high-frequency feature extraction based on image grayscale gradients. During the calibration of telecentric lenses or large depth-of-field systems, the calibration board must be moved at multiple positions along the optical axis to cover the entire measurement range. Due to the limitations of physical optics' depth-of-field principle, image defocusing is unavoidable. Defocusing is equivalent to low-pass filtering the image, causing significant attenuation and blurring of pixel grayscale gradients at corners. Traditional gradient-dependent algorithms struggle to accurately locate or even miss corners in blurred images, severely weakening the geometric robustness of calibration data across the entire depth-of-field range.
[0009] Second, the physical characteristics at the black-and-white boundary lead to phase calculation errors. In the projector calibration process, sinusoidal structured light stripes need to be projected onto the calibration plate, and the absolute phase at feature points needs to be accurately calculated to establish a mapping relationship. However, the corner points are physically located at the boundaries of the black and white squares in a checkerboard pattern, where there is an extreme step in surface reflectivity. When grating stripes are superimposed at this location, the low reflectivity of the black area and the abrupt change in edge light intensity cause the stripe modulation to be extremely low, even close to zero, resulting in a significant decrease in the signal-to-noise ratio. This physical defect easily induces singularities and phase jumps at the corner points, meaning that existing technologies use corner point phase data containing nonlinear errors and inaccurate calculations as calibration benchmarks, fundamentally limiting the improvement of system calibration accuracy. Summary of the Invention
[0010] To address the aforementioned problems in the existing technology, this invention provides a method and apparatus for calibrating a telecentric structured light system based on the geometric center of a checkerboard pattern. The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, embodiments of the present invention provide a method for calibrating a telecentric structured light system based on the geometric center of a checkerboard pattern, the method comprising: S1, Select a test calibration board image from the calibration board image set, perform image edge detection and binarization processing based on the Sobel operator to obtain a binarized edge map; wherein, the multiple calibration board images in the calibration board image set are obtained by taking pictures of the calibration board with a checkerboard pattern on the surface at different distances from the camera. S2, perform line detection on the binarized edge map based on Hough transform, calculate the rotation correction angle corresponding to the vertical line of the chessboard grid, and construct a rotation matrix using the rotation correction angle; S3, construct a standard checkerboard template based on ROI selection and edge expansion processing of the test calibration board image; S4, normalize each calibration board image to obtain the corresponding normalized calibration board image; perform template matching operation and sub-pixel-level center positioning on each normalized calibration board image and the standard checkerboard template, and finally determine the center coordinates of the checkerboard of the corresponding calibration board image as the feature point coordinates. S5, the rotation matrix is used to perform attitude correction on the feature point coordinates of each calibration board image, thereby performing telecentric structured light system calibration.
[0011] In one embodiment of the present invention, S1 includes: S11, The Sobel operator is used to calculate the gradient of the test calibration board image to obtain the gradient magnitude matrix G of each pixel; S12, normalize the gradient magnitude matrix G to obtain the normalized gradient magnitude matrix. S13, the normalized gradient magnitude matrix is binarized to obtain a binarized edge map.
[0012] In one embodiment of the present invention, S2 includes: S21, perform line detection on the binarized edge map based on Hough transform to obtain a set of lines; S22, Select the vertical lines of the chessboard pattern from the set of lines to obtain the set of vertical lines; S23, Calculate the arithmetic mean of the angles of all vertical lines in the set of vertical lines, and use it as the rotation correction angle; S24, construct a rotation matrix using the rotation correction angle.
[0013] In one embodiment of the present invention, S3 includes: S31, Obtain the rectangular region containing the target chessboard grid on the test calibration board image as the initial ROI region; wherein, the target chessboard grid is a complete white chessboard grid; S32, perform binarization and geometric moment analysis on the image within the initial ROI region, and calculate the geometric properties of the internal checkerboard, including the centroid and bounding box size; S33, perform edge expansion based on the geometric properties to construct a standard checkerboard template.
[0014] In one embodiment of the present invention, in step S4, each normalized calibration board image is matched with the standard checkerboard template using the following formula: ; in, For normalized calibration plate images; The standard chessboard template; This represents a two-dimensional discrete convolution operation; and These represent the height and width of the standard chessboard template, respectively. This indicates the position within the standard checkerboard template with the center of the template as the origin. Pixels at that location; Indicates the position in the normalized calibration plate image The convolution response value at that point. In one embodiment of the present invention, the sub-pixel-level center positioning process in S4 includes: The convolutional response field obtained by template matching operation is subjected to non-maximum suppression algorithm. By comparing the response intensity of each pixel with its 8 neighboring pixels, local response peak points are selected to obtain the initial integer pixel coordinates of the feature points; wherein, the feature points represent the geometric center points of the checkerboard. Based on the initial integer pixel coordinates of the feature points and the convolutional response field data obtained by template matching, a local calculation window of a preset size is constructed with the initial integer pixel coordinates of the feature points as the center. The pixels within the local calculation window are then processed to remove the background, thereby obtaining effective response intensity data. Based on the initial integer pixel coordinates of the feature points and the effective response intensity data, the sub-pixel offset of the feature points relative to their initial integer pixel centers is calculated using the gray-weighted centroid method model, thereby obtaining the sub-pixel center coordinates of the checkerboard grid as the feature point coordinates.
[0015] In one embodiment of the present invention, the formula used for background removal processing of pixels within a local calculation window is as follows: (2); in, Indicating window data The original response value of the location; Indicating window data The original response value of the location; for Effective response intensity after removing local background noise; Represents a local calculation window; This indicates taking the minimum value. In one embodiment of the present invention, the formula used to calculate the sub-pixel offset of a feature point relative to its initial integer pixel center is: (3); in, The feature point's coordinates relative to the initial integer pixel coordinates Subpixel-level offset; This represents the radius of the local calculation window.
[0016] In one embodiment of the present invention, the preset size is Pixel size.
[0017] Secondly, embodiments of the present invention provide a calibration method and apparatus for a telecentric structured light system based on the geometric center of a checkerboard pattern, the apparatus comprising: The binarized edge map calculation module is used to select a test calibration board image from the calibration board image set, perform image edge detection and binarization processing based on the Sobel operator, and obtain a binarized edge map; wherein, the multiple calibration board images in the calibration board image set are obtained by taking pictures of the calibration board with a checkerboard pattern on the surface at different distances from the camera. The rotation matrix construction module is used to perform line detection on the binarized edge map based on Hough transform, calculate the rotation correction angle corresponding to the vertical line of the chessboard grid, and construct the rotation matrix using the rotation correction angle; A standard checkerboard template construction module is used to construct a standard checkerboard template from the test calibration board image based on ROI selection and edge expansion processing. The feature point coordinate calculation module is used to normalize each calibration board image to obtain the corresponding normalized calibration board image; and to perform template matching operation and sub-pixel-level center positioning on each normalized calibration board image and the standard checkerboard template to finally determine the center coordinates of the checkerboard of the corresponding calibration board image as the feature point coordinates. The calibration module is used to perform attitude correction on the feature point coordinates of each calibration board image using the rotation matrix, thereby calibrating the telecentric structured light system.
[0018] The present invention has the following beneficial effects: (1) Robustness and high-precision positioning: This invention employs a convolutional matching strategy based on edge-extended templates. By leveraging the convolution integral effect, it effectively smooths random noise in the image, overcoming the shortcomings of traditional differential operators that are sensitive to defocus blur and high-frequency noise, thus significantly improving the robustness of feature extraction. Simultaneously, combined with a "coarse-to-fine" hierarchical localization mechanism, it introduces a background-removing gray-level weighted centroid method on top of the coarse convolutional localization. This effectively eliminates the interference of local background noise, improving the feature point extraction accuracy to the sub-pixel level and ensuring the accuracy of subsequent phase mapping.
[0019] (2) Attitude adaptation and ease of operation: This invention eliminates the dependence on high-frequency corner features, which are sensitive to defocusing and have low phase signal-to-noise ratio, found in traditional techniques. It utilizes the Sobel gradient operator and Hough transform for attitude correction, enabling precise calculation of any rotation angle of the calibration board within the field of view. Furthermore, it automatically corrects the feature point coordinates to the physical axis direction using a rotation matrix. This removes the mechanical restriction that the calibration board must be strictly horizontal or vertically placed in actual calibration, effectively eliminating coordinate system errors caused by manual tilting. While reducing the complexity of calibration operations, it significantly improves the system's adaptability to different testing environments. Attached Figure Description
[0020] Figure 1 This is a flowchart illustrating a method for calibrating a telecentric structured light system based on the geometric center of a checkerboard pattern, as provided in an embodiment of the present invention. Figure 2 This is a binarized edge map of the test calibration board image in an embodiment of the present invention; Figure 3 This is a diagram showing the Hough detection results in an embodiment of the present invention; Figure 4 This is a schematic diagram of the selected template and the processed template in an embodiment of the present invention; Figure 5 This is a schematic diagram of the final geometric center result of the chessboard grid in an embodiment of the present invention; Figure 6 This is a schematic diagram of a telecentric structured light system calibration device based on the geometric center of a checkerboard pattern, provided in an embodiment of the present invention. Detailed Implementation
[0021] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.
[0022] As will be understood by those skilled in the art, a telecentric structured light system mainly consists of a structured light projector and an imaging camera equipped with dual telecentric lenses. Unlike traditional industrial lenses, telecentric lenses employ a parallel optical path design, and their main optical characteristic is that, within a certain depth of field, the magnification of the image does not change with the object distance (i.e., perspective error is eliminated). This characteristic gives the system a large measurement depth of field, but it also leads to potential defocus blurring at the edges of the depth of field.
[0023] Specifically, this invention aims to solve the following problems: (1) Solve the feature extraction failure caused by defocus blur during telecentric lens calibration. During the calibration of a telecentric imaging system, to cover the entire measurement field, the calibration board must be moved and photographed at multiple positions within the depth of field along the optical axis. Due to the limited depth of field and physical optical characteristics of the telecentric lens, image defocusing and blurring inevitably occur when the calibration board moves to the edge of the depth of field. This defocusing effect causes degradation of high-frequency edge features at corners, which are relied upon by traditional techniques, and a significant attenuation of pixel grayscale gradients. This makes it difficult for gradient-based corner detection algorithms (such as Harris) to maintain sub-pixel positioning accuracy in blurred images, and may even result in missed feature points, severely limiting the geometric robustness of the calibration data across the entire depth of field.
[0024] (2) Limitations in the accuracy of corner phase calculation in projector calibration In the projector calibration stage of structured light systems, the phase-shifting method is typically used to establish the correspondence between camera and projector pixels. However, the corner points of a checkerboard pattern are essentially the boundaries between black and white areas, where the surface reflectivity (or light intensity) changes abruptly. At these boundaries, the modulation of the projected fringes drops sharply and is highly susceptible to environmental noise interference, easily generating phase singularities during phase calculation. This discontinuity and nonlinear error in the phase data leads to significant deviations in corner-based feature mapping, failing to meet the calibration requirements for high-precision 3D reconstruction.
[0025] In a first aspect, embodiments of the present invention provide a method for calibrating a telecentric structured light system based on the geometric center of a checkerboard pattern, such as... Figure 1As shown, the method may include the following steps: S1. Select a test calibration board image from the calibration board image set, perform image edge detection and binarization based on the Sobel operator, and obtain a binarized edge map; The calibration board images in the image set are multiple images of the calibration board with a checkerboard pattern on its surface, captured at different distances from the camera; the calibration board images are typically grayscale images. The process of obtaining the calibration board image set can be found in related technical explanations and will not be described here.
[0026] Specifically, S1 may include the following steps: S11, The Sobel operator is used to calculate the gradient of the test calibration board image to obtain the gradient magnitude matrix G of each pixel; Since the calibration board may rotate during placement, in order to accurately extract the edge straight line features of the chessboard, the Sobel operator is first used to calculate the gradient of the image.
[0027] The specific process involves filtering the image using Sobel convolution kernels in both the horizontal and vertical directions to obtain horizontal and vertical gradient components. Then, the gradient magnitude at each pixel location is calculated, resulting in a gradient magnitude matrix G of the same size as the original image. This process aims to highlight the dramatic changes in pixel grayscale at the edges of the checkerboard pattern in the image, laying the foundation for subsequent line detection.
[0028] S12, normalize the gradient magnitude matrix G to obtain the normalized gradient magnitude matrix. Normalize all values in the gradient magnitude matrix G, linearly mapping them to a fixed range of values, which can be set as needed, for example, [0, 255]. The purpose of this step is to eliminate the problem of different absolute magnitudes of gradient values caused by differences in illumination and contrast between different images, so that the subsequent binarization threshold setting has consistency and robustness.
[0029] S13, the normalized gradient magnitude matrix is binarized to obtain a binarized edge map.
[0030] A binarization threshold can be set as needed. Iterate through each pixel in the normalized gradient magnitude matrix; if its value is greater than or equal to the threshold, assign a foreground value (e.g., 1 or 255) to the corresponding position in the output image; otherwise, assign a background value (e.g., 0), thus obtaining a binarized edge map. See also... Figure 2 As shown.
[0031] This process transforms continuous gradient magnitude information into a binary image containing only edge and non-edge regions, greatly simplifying the image data and facilitating the direct extraction of straight-line features via subsequent Hough transform. In the binarized edge image, white pixels (foreground) represent potential checkerboard edges, and black pixels (background) represent non-edge regions.
[0032] S2, perform line detection on the binarized edge map based on Hough transform, calculate the rotation correction angle corresponding to the vertical line of the chessboard grid, and construct a rotation matrix using the rotation correction angle; This step uses Hough transform for line detection in order to accurately calculate the rotational attitude of the calibration plate in the field of view.
[0033] S2 may include the following steps: S21, perform line detection on the binarized edge map based on Hough transform to obtain a set of lines; To accurately identify the grid lines of a checkerboard pattern from a binarized edge map, a Hough transform is used to map pixels in the image space to a parameter space. Collinear points are then counted using an accumulator to detect straight lines in the image. This results in a set of lines containing parameters for multiple lines. ; Indicates the first The parameters of the lines, where Indicates the first The normal distance from the origin to the straight line Indicates the first The angle (usually in radians) between the normal to a straight line and the X-axis. For details on line detection using the Hough transform, please refer to the relevant technical explanations.
[0034] S22, Select the vertical lines of the chessboard pattern from the set of lines to obtain the set of vertical lines; Based on the geometric properties of the checkerboard pattern, its vertical lines should ideally be parallel to the vertical direction of the image (i.e., the corresponding vertical lines). Angle close to 90 degrees or (Radians). First, the angle in the line parameters. The judgment is then made within a suitable angle range (e.g., [0, π) or [-π / 2, π / 2)). Then, the angle search range is set to... Step size is The system uses a set angle threshold to filter out lines whose angle values are close to the vertical direction (e.g., the center value of the transformed range). These filtered lines are then identified as the vertical lines of the checkerboard pattern. See the detection results below. Figure 3 understand.
[0035] S23, Calculate the arithmetic mean of the angles of all vertical lines in the set of vertical lines, and use it as the rotation correction angle; Extract the angle of each vertical line in the set of vertical lines. To accurately calculate the overall tilt angle of the calibration plate, this invention statistically analyzes the angles of multiple detected vertical lines. Specifically, these angles are calculated. The arithmetic mean of the angles is used as the rotation correction angle, and can be denoted as: Calculating the arithmetic mean can effectively reduce the noise error that may exist in the detection of a single straight line, thus obtaining a more robust and representative estimate of the overall rotation angle, i.e., the rotation correction angle. This angle represents the degree to which the vertical lines of the checkerboard deviate from the ideal vertical direction on average in the current test calibration board image.
[0036] S24, construct a rotation matrix using the rotation correction angle.
[0037] To perform geometric correction of feature point coordinates on the image plane to eliminate tilt effects, a rotation matrix needs to be constructed. This rotation matrix is used to subsequently rotate the feature point coordinates in the reverse direction. Angles are used to correct feature points to directions parallel to the physical axes of the checkerboard (i.e., the ideal world coordinate system orientation). Based on the geometric relationships of rotation in a two-dimensional plane, a rotation matrix R can be constructed; for details on the construction process, please refer to the relevant technical explanations.
[0038] This invention introduces a pose adaptive correction mechanism based on the Sobel operator-Hough transform. The rotation angle is calculated by detecting edge lines, and a rotation matrix is constructed for subsequent correction of feature point coordinates. This eliminates the reliance on the strictly mechanically orthogonal placement of the calibration plate during the calibration process, improving the algorithm's adaptability to arbitrary orientations.
[0039] S3, construct a standard checkerboard template based on ROI selection and edge expansion processing of the test calibration board image; To improve the noise resistance of corner detection, this invention constructs a standard checkerboard template. A template extraction strategy combining human-computer interaction and morphological analysis is adopted.
[0040] S3 may include the following steps: S31, Obtain the rectangular region containing the target chessboard grid on the test calibration board image as the initial ROI (Region of Interest); wherein, the target chessboard grid is a complete white chessboard grid; To focus on valid checkerboard regions and eliminate interference from irrelevant backgrounds, the initial ROI can be determined manually on the test calibration board image. Specifically, the region containing the target checkerboard square is manually selected as the initial search area for feature extraction. The aim is to reduce computational cost and improve the accuracy and robustness of subsequent processing, ensuring that the analysis is performed only on valid calibration feature regions. The target checkerboard square can be a complete white checkerboard square located at the center of the test calibration board image.
[0041] S32, perform binarization and geometric moment analysis on the image within the initial ROI region, and calculate the geometric properties of the internal checkerboard, including the centroid and bounding box size; First, the image within the initial ROI region is binarized, separating the black and white squares of the checkerboard pattern into foreground (e.g., white squares) and background (e.g., black squares). Then, a geometric moment analysis algorithm is applied to analyze the connected regions. By calculating the geometric moments of the main connected regions (representing the checkerboard squares) in the binary image, the geometric properties can be accurately calculated, including: 1) Centroid: The center coordinates of all foreground pixels or the main connected regions. The calculated centroid coordinates can be expressed as... .
[0042] 2) Bounding box dimensions: The width and height of the smallest bounding rectangle that can completely enclose the internal valid checkerboard area (the entire white checkerboard). The calculated bounding box dimensions can be expressed as... , These are the minimum bounding rectangles at... Length in the direction.
[0043] This step aims to precisely quantify the actual position and size of the effective checkerboard grid within the ROI at the pixel level, providing an accurate geometric benchmark for constructing a standard checkerboard template.
[0044] S33, perform edge expansion based on the geometric properties to construct a standard checkerboard template.
[0045] To enhance robustness to edge effects and incorporate richer gradient information in subsequent convolutional matching, this invention introduces an edge extension strategy. Specifically, the operation is as follows: based on the bounding box calculated in step S32, its four sides are uniformly extended outwards by a preset pixel width, for example, by expanding outwards... The pixel width, i.e. Expand outwards in direction pixel width, Expand outwards in direction The pixel width.
[0046] After expansion, using the original centroid position as a reference, an image patch corresponding to this expanded rectangular region is cropped from the original test calibration board image. This image patch constitutes the constructed standard checkerboard template. Template It not only contains clear textures within the checkerboard pattern, but also background or edge transition information around it, and contains rich gradient information. This makes it more discriminative and robust when performing convolutional matching with normalized calibration board images.
[0047] Understandably, the standard checkerboard template It is an image matrix. The selected template and the processed template are as follows: Figure 4 As shown, Figure 4 The image on the right is the standard chessboard template.
[0048] S4, normalize each calibration board image to obtain the corresponding normalized calibration board image; perform template matching operation and sub-pixel-level center positioning on each normalized calibration board image and the standard checkerboard template, and finally determine the center coordinates of the checkerboard of the corresponding calibration board image as the feature point coordinates. The process of normalizing each calibration board image in the calibration board image set will not be described in detail here.
[0049] In S4, each normalized calibration board image is matched with the standard checkerboard template using the following formula: (1); in, For normalized calibration plate images; The standard chessboard template; This represents a two-dimensional discrete convolution operation; and These represent the height and width of the standard chessboard template, respectively. This indicates the position within the standard checkerboard template with the center of the template as the origin. Pixels at that location; Indicates the position in the normalized calibration plate image The convolution response value at that point.
[0050] Understandably, region-based convolution operations utilize all pixel information within the checkerboard pattern, effectively suppressing random noise through integral effects. The convolution response is optimal when the local image region has the highest overlap with the template. Find the local maximum value, which corresponds to the geometric center of the chessboard square. This step can be understood as a coarse localization of the center position of a chessboard square.
[0051] To achieve sub-pixel accuracy, this invention employs a "coarse-to-fine" hierarchical positioning strategy. Building upon coarse convolutional positioning, a gray-level weighted centroid method is introduced for sub-pixel-level coordinate refinement. The sub-pixel-level center positioning process may include the following steps: 1) Apply nonmaximum suppression algorithm to the convolutional response field obtained by template matching operation, and filter out local response peak points by comparing the response intensity of each pixel with its 8 neighboring pixels to obtain the initial integer pixel coordinates of the feature points; wherein, the feature points represent the geometric center point of the checkerboard grid. 2) Based on the initial integer pixel coordinates of the feature points and the convolutional response field data obtained by template matching operation, a local calculation window of a preset size is constructed with the initial integer pixel coordinates of the feature points as the center. The pixels within the local calculation window are then processed to remove the background, thereby obtaining effective response intensity data. 3) Based on the initial integer pixel coordinates of the feature points and the effective response intensity data, and using the gray-weighted centroid model, calculate the sub-pixel offset of the feature points relative to their initial integer pixel centers, thereby obtaining the sub-pixel center coordinates of the checkerboard as the feature point coordinates.
[0052] Specifically, firstly, using a non-maximum suppression algorithm, the local response peak is selected by comparing the response intensity of the center pixel with its 8 neighboring pixels, and this peak is then used as the initial integer pixel coordinates of the feature point (i.e., the geometric center point of the checkerboard). Subsequently, a local calculation window is constructed centered on this coordinate. Considering that the response exhibits a Gaussian-like distribution near the peak, in order to preserve the core peak information and suppress background noise, this invention selects... The pixel area is pre-processed to remove the background from the data within the window using the following formula: (2); in, This indicates the data within the window (data within the local computation window). The original response value of the location; Indicating window data The original response value of the location; for Effective response intensity after removing local background noise; Represents a local calculation window; This indicates taking the minimum value. Based on the gray-level centroid method model, the sub-pixel offset of a feature point relative to its initial integer pixel coordinates. It can be calculated as: (3); in, The feature point's coordinates relative to the initial integer pixel coordinates Subpixel-level offset; This represents the radius of the local calculation window.
[0053] Finally, the corrected sub-pixel level center coordinates are Compared to traditional corner detection, this method is more robust to blurred and out-of-focus image edges, and the detected checkerboard geometric center results are as follows: Figure 5 As shown.
[0054] The embodiments of this invention adopt a "convolution coarse localization-centroid fine localization" anti-defocus strategy. It first uses two-dimensional convolution (template matching) based on region integral to perform initial feature localization, which effectively smooths the gradient diffusion caused by defocus blur. Then, it combines gray-scale weighted centroid algorithm to perform sub-pixel refinement, which can ensure that even if the image edges are blurred, the center coordinates can be accurately extracted within the telecentric large depth of field range.
[0055] S5, the rotation matrix is used to perform attitude correction on the feature point coordinates of each calibration board image, thereby performing optical system calibration.
[0056] It is understandable that the coordinates of the feature points (i.e., the image pixel coordinates of the geometric center point) of each chessboard grid in each calibration board image are obtained through S4.
[0057] The rotation matrix obtained in S2 is used to correct the coordinates of the extracted feature points to a direction parallel to the physical axis of the chessboard, thereby eliminating the coordinate error introduced by the tilt of the calibration board and obtaining a set of corrected feature point coordinates with a unified orientation reference after attitude correction.
[0058] Based on the set of coordinates of the calibration feature points corresponding to all calibration board images and their known three-dimensional coordinates in the calibration board's physical coordinate system (world coordinate system), the calibration of the optical system (mainly referring to the camera and projector) is performed using the calibrated feature point coordinates.
[0059] In short, the calibration process involves constructing a system of linear equations containing the correspondences of feature points based on an established world coordinate system. Solving this system determines the initial calibration matrices for the camera and projector. For example, for a camera, the process mainly involves constructing a system of linear equations containing camera intrinsic parameters (focal length, principal point, distortion, etc.) and extrinsic parameters (rotation, translation) based on the established world coordinate system and the correspondence between each feature point's coordinates in the image pixel coordinate system (corrected coordinates) and its 3D coordinates in the world coordinate system. Solving this system of equations using optimization algorithms such as least squares determines the initial calibration matrix (i.e., the projection matrix) for the camera. Subsequently, nonlinear optimization (such as Bundle Adjustment based on reprojection error) is typically performed to further refine all parameters. For details on the camera and projector calibration process, please refer to relevant technical explanations; further details are not provided here.
[0060] Comparing traditional corner detection with the geometric center feature localization method of this invention, in terms of camera calibration, compared to the traditional corner detection method which has poor stability and requires multiple shots for complete identification, the template matching feature localization method used in this invention reduces the camera reprojection error by 0.286 pixels. In terms of projector calibration, addressing the problem of high computational noise due to low modulation at traditional corners, this invention proposes using the geometric center of a high-reflectivity checkerboard pattern instead of corners for phase extraction, reducing the feature point reprojection error by 0.262 pixels.
[0061] To address the insufficient noise resistance of corner features in traditional structured light calibration, particularly the low phase accuracy caused by drastic reflectivity changes at the black-and-white boundary, this invention provides a telecentric structured light system calibration method based on the geometric center of a checkerboard pattern. This method is a highly robust checkerboard geometric center calibration scheme based on adaptive template convolution. Traditional calibration relies on corner points located in reflectivity step regions, which are affected by defocusing blur and edge response. The fringe modulation at these points is extremely low, easily leading to phase unwrapping singularities and phase jumps, severely limiting calibration accuracy.
[0062] The present invention has the following beneficial effects: (1) Robustness and high-precision positioning: This invention employs a convolutional matching strategy based on edge-extended templates. By leveraging the convolution integral effect, it effectively smooths random noise in the image, overcoming the shortcomings of traditional differential operators that are sensitive to defocus blur and high-frequency noise, thus significantly improving the robustness of feature extraction. Simultaneously, combined with a "coarse-to-fine" hierarchical localization mechanism, it introduces a background-removing gray-level weighted centroid method on top of the coarse convolutional localization. This effectively eliminates the interference of local background noise, improving the feature point extraction accuracy to the sub-pixel level and ensuring the accuracy of subsequent phase mapping.
[0063] (2) Attitude adaptation and ease of operation: This invention eliminates the dependence on high-frequency corner features, which are sensitive to defocusing and have low phase signal-to-noise ratio, found in traditional techniques. It utilizes the Sobel gradient operator and Hough transform for attitude correction, enabling precise calculation of any rotation angle of the calibration board within the field of view. Furthermore, it automatically corrects the feature point coordinates to the physical axis direction using a rotation matrix. This removes the mechanical restriction that the calibration board must be strictly horizontal or vertically placed in actual calibration, effectively eliminating coordinate system errors caused by manual tilting. While reducing the complexity of calibration operations, it significantly improves the system's adaptability to different testing environments.
[0064] Secondly, corresponding to the above method embodiments, this invention also provides a telecentric structured light system calibration device based on the geometric center of a checkerboard pattern, such as... Figure 6 As shown, the device includes: The binarized edge map calculation module is used to select a test calibration board image from the calibration board image set, perform image edge detection and binarization processing based on the Sobel operator, and obtain a binarized edge map; wherein, the multiple calibration board images in the calibration board image set are obtained by taking pictures of the calibration board with a checkerboard pattern on the surface at different distances from the camera. The rotation matrix construction module is used to perform line detection on the binarized edge map based on Hough transform, calculate the rotation correction angle corresponding to the vertical line of the chessboard grid, and construct the rotation matrix using the rotation correction angle; A standard checkerboard template construction module is used to construct a standard checkerboard template from the test calibration board image based on ROI selection and edge expansion processing. The feature point coordinate calculation module is used to normalize each calibration board image to obtain the corresponding normalized calibration board image; and to perform template matching operation and sub-pixel-level center positioning on each normalized calibration board image and the standard checkerboard template to finally determine the center coordinates of the checkerboard of the corresponding calibration board image as the feature point coordinates. The calibration module is used to perform attitude correction on the feature point coordinates of each calibration board image using the rotation matrix, thereby calibrating the telecentric structured light system.
[0065] For details on the specific processing procedures of each module of the device, please refer to the relevant content in the first section, which will not be repeated here.
[0066] This invention aims to solve the problems of unstable feature matching and high susceptibility to environmental noise interference in the calibration process of cameras and projectors in existing structured light 3D measurement systems. The solution provided by this invention abandons the corner features, which are sensitive to defocusing and have low phase signal-to-noise ratio, found in traditional techniques. It utilizes the Sobel gradient operator and Hough transform for fusion attitude adaptive correction, and employs adaptive template convolution and sub-pixel positioning strategies to accurately extract the geometric center of a checkerboard pattern with uniform light intensity and continuous phase. By leveraging the regional integration effect of convolution operations to overcome edge blurring interference in telecentric imaging, and avoiding reflectivity step noise at the black-and-white boundary, phase unwrapping errors are fundamentally eliminated, significantly improving the calibration accuracy and 3D reconstruction stability of the structured light system in complex environments. Because the checkerboard center region has uniform light intensity distribution, continuous and smooth phase, this solution effectively avoids phase noise interference at the edges, significantly improving the system calibration stability and 3D measurement accuracy in small field-of-view environments.
[0067] It should be noted that, in the description of this invention, the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0068] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0069] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.
[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A calibration method for a telecentric structured light system based on the geometric center of a checkerboard pattern, characterized in that, include: S1, Select a test calibration board image from the calibration board image set, perform image edge detection and binarization processing based on the Sobel operator to obtain a binarized edge map; wherein, the multiple calibration board images in the calibration board image set are obtained by taking pictures of the calibration board with a checkerboard pattern on the surface at different distances from the camera. S2, perform line detection on the binarized edge map based on Hough transform, calculate the rotation correction angle corresponding to the vertical line of the chessboard grid, and construct a rotation matrix using the rotation correction angle; S3, construct a standard checkerboard template based on ROI selection and edge expansion processing of the test calibration board image; S4, normalize each calibration board image to obtain the corresponding normalized calibration board image; perform template matching operation and sub-pixel-level center positioning on each normalized calibration board image and the standard checkerboard template, and finally determine the center coordinates of the checkerboard of the corresponding calibration board image as the feature point coordinates. S5, the rotation matrix is used to perform attitude correction on the feature point coordinates of each calibration board image, thereby performing telecentric structured light system calibration.
2. The calibration method for a telecentric structured light system based on the geometric center of a checkerboard pattern according to claim 1, characterized in that, S1 includes: S11, The Sobel operator is used to calculate the gradient of the test calibration board image to obtain the gradient magnitude matrix G of each pixel; S12, normalize the gradient magnitude matrix G to obtain the normalized gradient magnitude matrix. S13, the normalized gradient magnitude matrix is binarized to obtain a binarized edge map.
3. The calibration method for a telecentric structured light system based on the geometric center of a checkerboard pattern according to claim 1, characterized in that, S2 includes: S21, perform line detection on the binarized edge map based on Hough transform to obtain a set of lines; S22, Select the vertical lines of the chessboard pattern from the set of lines to obtain the set of vertical lines; S23, Calculate the arithmetic mean of the angles of all vertical lines in the set of vertical lines, and use it as the rotation correction angle; S24, construct a rotation matrix using the rotation correction angle.
4. The calibration method for a telecentric structured light system based on the geometric center of a checkerboard pattern according to claim 1, characterized in that, S3 includes: S31, Obtain the rectangular region containing the target chessboard grid on the test calibration board image as the initial ROI region; wherein, the target chessboard grid is a complete white chessboard grid; S32, perform binarization and geometric moment analysis on the image within the initial ROI region, and calculate the geometric properties of the internal checkerboard, including the centroid and bounding box size; S33, perform edge expansion based on the geometric properties to construct a standard checkerboard template.
5. The calibration method for a telecentric structured light system based on the geometric center of a checkerboard pattern according to claim 1, characterized in that, In S4, each normalized calibration board image is matched with the standard checkerboard template using the following formula: ； in, For normalized calibration plate images; The standard chessboard template; This represents a two-dimensional discrete convolution operation; and These represent the height and width of the standard chessboard template, respectively. This indicates the position within the standard checkerboard template with the center of the template as the origin. Pixels at that location; Indicates the position in the normalized calibration plate image The convolution response value at that point.
6. The calibration method for a telecentric structured light system based on the geometric center of a checkerboard pattern according to claim 5, characterized in that, The process of sub-pixel-level center positioning in S4 includes: The convolutional response field obtained by template matching operation is subjected to non-maximum suppression algorithm. By comparing the response intensity of each pixel with its 8 neighboring pixels, local response peak points are selected to obtain the initial integer pixel coordinates of the feature points; wherein, the feature points represent the geometric center points of the checkerboard. Based on the initial integer pixel coordinates of the feature points and the convolutional response field data obtained by template matching, a local calculation window of a preset size is constructed with the initial integer pixel coordinates of the feature points as the center. The pixels within the local calculation window are then processed to remove the background, thereby obtaining effective response intensity data. Based on the initial integer pixel coordinates of the feature points and the effective response intensity data, the sub-pixel offset of the feature points relative to their initial integer pixel centers is calculated using the gray-weighted centroid method model, thereby obtaining the sub-pixel center coordinates of the checkerboard grid as the feature point coordinates.
7. The calibration method for a telecentric structured light system based on the geometric center of a checkerboard pattern according to claim 6, characterized in that, The formula used to remove background from pixels within a local calculation window is: (2)； in, Indicating window data The original response value of the location; Indicating window data The original response value of the location; for Effective response intensity after removing local background noise; Represents a local calculation window; This indicates taking the minimum value.
8. The calibration method for a telecentric structured light system based on the geometric center of a checkerboard pattern according to claim 7, characterized in that, The formula used to calculate the sub-pixel offset of a feature point relative to its initial integer pixel center is: (3)； in, The feature point's coordinates relative to the initial integer pixel coordinates Subpixel-level offset; This represents the radius of the local calculation window.
9. The calibration method for a telecentric structured light system based on the geometric center of a checkerboard pattern according to claim 6, characterized in that, The preset size is Pixel size.
10. A calibration device for a telecentric structured light system based on the geometric center of a checkerboard pattern, characterized in that, include: The binarized edge map calculation module is used to select a test calibration board image from the calibration board image set, perform image edge detection and binarization processing based on the Sobel operator, and obtain a binarized edge map; wherein, the multiple calibration board images in the calibration board image set are obtained by taking pictures of the calibration board with a checkerboard pattern on the surface at different distances from the camera. The rotation matrix construction module is used to perform line detection on the binarized edge map based on Hough transform, calculate the rotation correction angle corresponding to the vertical line of the chessboard grid, and construct the rotation matrix using the rotation correction angle; A standard checkerboard template construction module is used to construct a standard checkerboard template from the test calibration board image based on ROI selection and edge expansion processing. The feature point coordinate calculation module is used to normalize each calibration board image to obtain the corresponding normalized calibration board image; and to perform template matching operation and sub-pixel-level center positioning on each normalized calibration board image and the standard checkerboard template to finally determine the center coordinates of the checkerboard of the corresponding calibration board image as the feature point coordinates. The calibration module is used to perform attitude correction on the feature point coordinates of each calibration board image using the rotation matrix, thereby calibrating the telecentric structured light system.