Calibration board pose generation and screening method based on geometric constraints and related equipment

Abstract

This application provides a method and related equipment for generating and selecting calibration board poses based on geometric constraints. In this method, the camera calibration system first acquires the hardware parameters of the camera, lens, and calibration board. It then uses a thin lens model combined with lens parameters to obtain the near-far distance of the depth of field, and further combines this with the field of view to determine the three-dimensional feasible space range of the calibration board. Next, the camera calibration system samples depth coordinates within the depth of field range, calculates the maximum allowable tilt angle and generates a rotation matrix, while simultaneously sampling translation vectors to construct temporary poses and calculate corner coordinates. Then, the camera calibration system eliminates temporary poses that exceed the feasible space at corner points and those with high similarity to existing poses, obtaining a set of candidate poses. Finally, the camera calibration system calculates a comprehensive distance metric for the candidate poses and iteratively selects the pose with the largest distance to form the target pose set. This method alleviates the technical problem of camera parameter calculations deviating from the true values ​​and improves the accuracy of camera calibration.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and related equipment for generating and screening calibration board poses based on geometric constraints. Background Technology

[0002] Camera calibration is a fundamental step in computer vision and precision measurement systems. Its purpose is to establish the mapping relationship between the three-dimensional coordinates of an object and the two-dimensional coordinates of an image. It usually requires controlling the calibration plate to undergo multiple transformations within the camera's field of view to acquire a set of image data containing rich position and attitude information. The geometric constraints of this data are then used to solve for the camera's intrinsic and extrinsic parameters.

[0003] In related technologies, camera calibration mainly focuses on the post-processing of existing data. Specifically, this method typically involves selecting a subset of high-quality images from the acquired calibration board image set after image acquisition, based on specific algorithmic metrics (such as reprojection error or confidence level of corner point extraction). Finally, this selected and retained data is used for camera parameter calibration.

[0004] However, this method of post-processing and filtering existing data results in a final dataset that is entirely dependent on the randomness of the initial acquisition. Since the filtering process is typically based on quality metrics of a single frame (such as reprojection error), this approach tends to retain images with better imaging quality within one or a few specific regions. This leads to the final calibration dataset often exhibiting spatial convergence and uniform pose distribution. Such a dataset with extremely high feature redundancy causes the calculated camera parameters to deviate from their true values, making it difficult to achieve the required calibration accuracy for high-precision measurements. Summary of the Invention

[0005] This application provides a method and related equipment for generating and filtering calibration plate poses based on geometric constraints, which can alleviate the technical problem of camera parameter calculation deviating from the true value and improve the accuracy of camera calibration.

[0006] Firstly, this application provides a calibration plate pose generation and selection method based on geometric constraints, applied to a camera calibration system. The camera calibration system includes a camera, a lens, and a calibration plate. The method includes: acquiring hardware parameters of the camera, lens, and calibration plate respectively; inputting the lens hardware parameters into a thin lens model to obtain the near-field distance and far-field distance of the camera at a preset focusing distance; defining a plane located at the near-field distance and perpendicular to the camera's optical axis as the front section, and a plane located at the far-field distance and perpendicular to the camera's optical axis as the rear section; determining the field of view angle based on the camera's hardware parameters, and constructing the side boundary of the view frustum based on the field of view angle; using the area enclosed by the front section, rear section, and side boundary as the three-dimensional feasible space range of the calibration plate; sampling the depth coordinates of the calibration plate between the near-field distance and the far-field distance, and based on the depth coordinates and the calibration plate... The hardware parameters of the calibration board are determined, and the maximum allowable tilt angle of the calibration board normal vector relative to the camera optical axis is calculated. Under the constraint of the maximum allowable tilt angle, the rotation matrix of the calibration board is generated, and the translation vector of the calibration board is sampled within the field of view corresponding to the depth coordinates. A temporary pose is constructed based on the rotation matrix and translation vector, and the corner coordinates of the calibration board in the temporary pose are calculated based on the hardware parameters of the calibration board. Temporary poses whose corner coordinates exceed the range of the three-dimensional feasible space are eliminated, and temporary poses with similarity to existing poses higher than a preset threshold are also eliminated to obtain a set of candidate poses. The translation distance metric and rotation distance metric between each candidate pose in the candidate pose set are calculated to obtain a comprehensive distance metric. Based on the comprehensive distance metric, the candidate pose with the largest distance to the selected pose set is iteratively selected and added to the target pose set until the number of poses in the target pose set reaches a preset number.

[0007] By adopting the above technical solution, the camera calibration system first acquires the hardware parameters of the camera, lens, and calibration board. The lens hardware parameters are input into a thin lens model to obtain the near and far distances of the depth of field. Combined with the field of view angle, the side boundary of the viewing cone is constructed, thereby determining the three-dimensional feasible space range of the calibration board. This range clearly defines the effective area where the calibration board can achieve clear imaging and is within the camera's field of view. Then, the camera calibration system samples depth coordinates within the depth of field range and calculates the maximum allowable tilt angle based on the calibration board hardware parameters. Next, under the constraint of the maximum allowable tilt angle, the camera calibration system generates a rotation matrix, samples translation vectors within the corresponding field of view section, constructs a temporary pose, and calculates corner coordinates. Through double rejection (corner points exceeding the feasible space, or poses with excessive similarity to existing poses), a set of candidate poses is obtained, eliminating invalid and duplicate poses. Finally, the camera calibration system calculates translational and rotational distance metrics between candidate poses and obtains a comprehensive distance metric through normalized weighting. Based on this metric, iteratively selects the poses with the largest distances to form the target pose set, ensuring the uniformity and diversity of the target poses in spatial distribution. This alleviates the problems of pose convergence and homogeneity that occur in post-processing selection. This method allows the pose set used for calibration to cover the camera's field of view and depth of field, possessing rich position and attitude information, thereby improving the calculation accuracy of camera intrinsic and extrinsic parameters, and ultimately enhancing the overall accuracy of camera calibration.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the translational distance metric and rotational distance metric between each candidate pose in the candidate pose set are calculated respectively to obtain a comprehensive distance metric. Specifically, this includes: calculating the translational distance metric and rotational distance metric between each candidate pose in the candidate pose set respectively; obtaining the translational distance between any two poses in the candidate pose set, and determining the translational distance with the largest value as the translational normalization scale, and determining the theoretical maximum allowable relative rotation angle between any two poses as the rotational normalization scale; normalizing the translational distance metric using the translational normalization scale, and normalizing the rotational distance metric using the rotational normalization scale; and weighting and summing the normalized translational distance metric and the normalized rotational distance metric according to preset weight coefficients to obtain a comprehensive distance metric.

[0009] By employing the above technical solution, the camera calibration system calculates translational and rotational distance metrics to determine the differences in pose in terms of position and angle. Then, by determining a normalization scale, the dimensional differences between the two metrics are eliminated. After normalization, the distance is weighted and summed according to preset weights to obtain a comprehensive distance metric. This metric reflects the overall differences between candidate poses, alleviates the limitations of single-dimensional metrics, and improves the uniformity and diversity of target pose distribution.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, based on a comprehensive distance metric, the candidate pose with the largest distance to the selected pose set is iteratively selected and added to the target pose set until the number of poses in the target pose set reaches a preset number. Specifically, this includes: calculating the average distance between each candidate pose in the candidate pose set and all other candidate poses based on the comprehensive distance metric; selecting the candidate pose with the largest average distance as the initial pose and adding the initial pose to the target pose set; for each remaining candidate pose in the candidate pose set, calculating the comprehensive distance metric between each candidate pose and all poses in the target pose set, and selecting the minimum value of the comprehensive distance metric as the closest distance between the candidate pose and the target pose set; comparing the closest distances corresponding to all candidate poses, and selecting the candidate pose corresponding to the closest distance with the largest value to add to the target pose set; repeating the process of calculating the closest distance and selecting the largest value until the number of poses in the target pose set reaches a preset number.

[0011] By adopting the above technical solution, the camera calibration system first selects the candidate pose with the largest average distance as the initial pose, then iteratively calculates the nearest distance between the remaining candidate poses and the target pose set, and selects the pose with the largest nearest distance to add to the set, until a preset number is reached. This process, through gradual optimization, ensures that the target pose set achieves the most uniform spatial distribution and maximizes pose differences within a finite number, avoiding pose convergence and improving the accuracy of camera parameter calculation.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, based on depth coordinates and hardware parameters of the calibration board, the maximum permissible tilt angle of the calibration board normal vector relative to the camera optical axis is calculated, specifically including: calculating the diagonal length of the calibration board according to the width and height of the calibration board; calculating a first distance difference between the depth coordinates and the near-end distance of the depth of field, calculating a second distance difference between the depth coordinates and the far-end distance of the depth of field, and selecting the minimum value of the first distance difference and the second distance difference as the depth tolerance distance; calculating a first tilt angle limited to the depth of field range using trigonometric relationships based on the diagonal length and the depth tolerance distance, the depth of field range being determined based on the near-end distance and the far-end distance of the depth of field; obtaining a preset maximum feature-recognizable tilt angle allowed by the camera for imaging, as the second tilt angle, and determining the minimum value of the first tilt angle and the second tilt angle as the maximum permissible tilt angle of the calibration board normal vector relative to the camera optical axis.

[0013] By employing the above technical solution, the camera calibration system, combining the calibration board size and the current depth tolerance distance, calculates the first tilt angle constrained by depth of field using trigonometric relationships. This is then combined with the camera's preset maximum identifiable tilt angle (second tilt angle), and the minimum of the two is taken as the maximum permissible tilt angle. This method simultaneously considers both clear depth-of-field imaging and corner feature identifiability, providing reasonable constraints for rotation matrix generation and ensuring the effectiveness of temporary poses.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, under the constraint of the maximum allowable tilt angle, generating the rotation matrix of the calibration plate specifically includes: constructing a conical space with the camera optical axis as the central axis and the maximum allowable tilt angle as the semi-vertical angle; randomly sampling the direction angle and tilt angle of the calibration plate normal vector within the conical space, and determining the intermediate normal vector based on the direction angle and tilt angle; based on the intermediate normal vector and the direction vector of the camera optical axis, using the principle of axis-angle rotation transformation, constructing a first rotation matrix that rotates the camera optical axis to coincide with the intermediate normal vector; randomly sampling the rotation angle of the calibration plate around its own normal, and constructing a second rotation matrix based on the rotation angle; multiplying the first rotation matrix and the second rotation matrix to obtain the rotation matrix.

[0015] By adopting the above technical solution, the camera calibration system samples the normal vector within the conical space constrained by the maximum allowable tilt angle, constructs a first rotation matrix that rotates the camera optical axis to this normal vector, and then constructs a second rotation matrix by combining it with the rotation angle of the calibration board. Multiplying the two matrices yields the final rotation matrix. This process ensures that the rotation matrix conforms to the tilt constraint and increases attitude diversity through rotation sampling, providing support for the diversity of temporary poses.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, temporary poses whose corner coordinates exceed the range of the three-dimensional feasible space are removed, and temporary poses whose similarity to existing poses is higher than a preset threshold are also removed to obtain a candidate pose set. Specifically, this includes: removing temporary poses whose corner coordinates exceed the range of the three-dimensional feasible space, and projecting the corner coordinates of the retained temporary poses onto the pixel plane according to the camera's hardware parameters to obtain pixel coordinates; removing temporary poses whose pixel coordinates exceed the imaging range of the image sensor to obtain a pose to be compared; calculating the translation distance difference between the pose to be compared and the existing poses, and when the translation distance difference is less than a preset translation threshold, calculating the relative rotation angle between the pose to be compared and the existing poses; removing poses to be compared whose relative rotation angle is less than a preset rotation threshold, and adding the remaining poses to be compared to the candidate pose set.

[0017] By adopting the above technical solution, the camera calibration system first eliminates temporary poses where corner points exceed the feasible space and pixels exceed the imaging range. Then, through dual comparison of translation distance difference and relative rotation angle, it eliminates poses that are excessively similar to existing poses, thus obtaining a set of candidate poses. This screening process ensures the effectiveness and diversity of candidate poses, eliminates invalid and duplicate data, and provides a high-quality foundation for target pose selection.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, the hardware parameters of the lens are input into a thin lens model to obtain the near-end distance and far-end distance of the depth of field at a preset focusing distance. Specifically, this includes: inputting the hardware parameters of the lens into a thin lens model; calculating the hyperfocal distance of the lens based on the lens's focal length, aperture number, and permissible circle of confusion diameter, wherein the hyperfocal distance is directly proportional to the square of the focal length and inversely proportional to the product of the aperture number and the permissible circle of confusion diameter; and calculating the near-end distance and far-end distance of the depth of field based on the hyperfocal distance and the preset focusing distance.

[0019] By adopting the above technical solution, the camera calibration system calculates the hyperfocal distance based on a thin lens model and hardware parameters such as lens focal length and aperture number. This hyperfocal distance is then combined with a preset focusing distance to obtain the near and far depth of field. This calculation, based on the lens imaging principle and the actual calibration scenario, accurately defines the depth range of the calibration board's clear image, providing accurate constraints for the construction of a three-dimensional feasible space and ensuring the effectiveness of subsequent pose generation.

[0020] In a second aspect, this application provides a camera calibration system, including a camera, a lens, and a calibration plate; the camera calibration system includes one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the camera calibration system to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, this application provides a computer-readable storage medium including instructions that, when executed on a camera calibration system, cause the camera calibration system to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, this application provides a computer program product, including a computer program / instruction that, when run on a camera calibration system, causes the camera calibration system to perform the method described in the first aspect and any possible implementation thereof.

[0023] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0024] 1. By adopting a technical approach that determines the three-dimensional feasible space based on hardware parameters, generates poses by combining the maximum allowable tilt angle constraint, and obtains a uniformly distributed set of target poses through screening and optimization, the technical problem of camera parameter calculation deviating from the true value due to pose convergence and single distribution is effectively alleviated. This achieves the technical effect of improving camera calibration accuracy and providing reliable pose support for high-precision calibration.

[0025] 2. By adopting a technical approach that combines the calibration plate size, depth tolerance, and camera imaging requirements to determine the maximum allowable tilt angle of the calibration plate normal vector, the technical problem of blurred imaging or indistinguishable corner points caused by excessive calibration plate tilt is effectively alleviated. This achieves the technical effect of ensuring the effectiveness of temporary pose and providing reasonable constraints for the generation of rotation matrix.

[0026] 3. By employing the technique of sampling the normal vector under tilt constraints and combining it with the rotation angle to construct the calibration plate rotation matrix, the technical problems of unconstrained rotation matrix and single posture are effectively alleviated, thereby achieving a technical effect that balances the effectiveness and diversity of pose. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a calibration plate pose generation and selection method based on geometric constraints in an embodiment of this application.

[0028] Figure 2 This is a schematic diagram of the optimal imaging cone in an embodiment of this application;

[0029] Figure 3 This is another flowchart illustrating the calibration plate pose generation and screening method based on geometric constraints in the embodiments of this application;

[0030] Figure 4 This is a schematic diagram of a conical space constrained by a normal vector in an embodiment of this application;

[0031] Figure 5 This is a schematic diagram of the physical device structure of a camera calibration system in an embodiment of this application. Detailed Implementation

[0032] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0033] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0034] The following describes the process of the method provided in this implementation. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating a calibration plate pose generation and selection method based on geometric constraints in an embodiment of this application.

[0035] 101. Obtain the hardware parameters of the camera, lens, and calibration board respectively. Input the hardware parameters of the lens into the thin lens model to obtain the near-field distance and far-field distance of the camera at the preset focus distance.

[0036] Hardware parameters represent a set of numerical values ​​used to describe the physical characteristics of the imaging system and calibration reference. They refer to the inherent attribute data necessary for calibration and capable of affecting the imaging geometry and depth of field. For example, camera hardware parameters may include sensor size, resolution, and pixel physical size; lens hardware parameters may include focal length, aperture number, field of view, focusing distance, and allowable circle of confusion diameter; and calibration plate hardware parameters may include the physical dimensions of the calibration plate, the number of corner points, and their spacing. A thin lens model is an idealized optical imaging mathematical model used to represent the imaging law after light is refracted through a lens. The preset focus distance refers to the distance from the current focusing plane of the camera lens to the optical center of the camera, used to represent the object distance position where the image is sharpest. The near-field distance refers to the closest distance boundary to the camera where the image remains sharp at the current focus distance; the far-field distance refers to the farthest distance boundary from the camera where the image remains sharp at the current focus distance.

[0037] Specifically, before initiating the calibration process, the camera calibration system needs to collect and process basic parameter information of the relevant hardware devices. First, the system acquires the camera's hardware parameters through the device interface or user input, including key technical indicators such as image sensor resolution, pixel size, and effective imaging area size. Next, it acquires the lens's hardware parameters, focusing on optical characteristics such as the lens's nominal focal length, current aperture value, and permissible circle of confusion diameter. Simultaneously, the system also needs to acquire the physical specifications of the calibration board, including its length, width, and feature point spacing. After collecting these hardware parameters, the system substitutes the lens's key optical parameters into a thin lens theoretical model for calculation. Based on the lens's focal length, aperture number, and permissible circle of confusion diameter, the system calculates the lens's hyperfocal length using mathematical formulas from the thin lens model. This hyperfocal length is directly proportional to the square of the focal length and inversely proportional to the product of the aperture number and the permissible circle of confusion diameter. Based on the calculated hyperfocal distance and the preset focus distance, the camera calibration system further calculates the depth-of-field range boundary under the current focus setting, namely the near-end distance and the far-end distance of the depth of field. These two distance values ​​will serve as the front and rear boundary constraints for the subsequent construction of the view frustum.

[0038] It should be noted that during subsequent camera calibration image acquisition, the lens's focus ring should typically remain fixed at the preset focus distance, or the autofocus function should be turned off, to ensure that the camera's intrinsic parameters (especially focal length) do not drift during acquisition. The preset focus distance is usually set to the distance at which the calibration board achieves the clearest image in the center of the field of view, or the working distance of the camera in its actual operating environment.

[0039] In some embodiments, the depth of field range can be determined as follows: the camera calibration system substitutes the lens's focal length f (mm), aperture number N, and allowable circle of confusion diameter coc (mm) into the thin lens hyperfocal distance calculation formula, and calculates the lens's hyperfocal distance H through mathematical relationships:

[0040]

[0041] The square of the focal length reflects the quadratic effect of the focal length on the depth of field, while the product of the aperture number and the diameter of the circle of confusion reflects the constraint of the aperture setting on the image sharpness requirement. Subsequently, the camera calibration system, based on the numerical relationship between the hyperfocal distance H and the preset focus distance s, calculates the near-field distance using the formula... Formula for calculating the distance between the far end and the center of the field The distances between the front and rear boundaries of the depth of field were calculated separately:

[0042]

[0043]

[0044] These two distance values ​​precisely define the spatial range in which acceptable image sharpness can be obtained with the current lens settings.

[0045] 102. Determine the plane located at the near end of the depth of field and perpendicular to the camera's optical axis as the front section, and the plane located at the far end of the depth of field and perpendicular to the camera's optical axis as the rear section. Determine the field of view angle based on the camera's hardware parameters, and construct the side boundary of the field of view cone based on the field of view angle.

[0046] The front section represents a plane located at the near end of the depth of field in the camera coordinate system and perpendicular to the camera's optical axis. This plane marks the closest effective position where the calibration plate can be placed. The rear section represents a plane located at the far end of the depth of field in the camera coordinate system and perpendicular to the camera's optical axis. This plane marks the farthest effective position where the calibration plate can be placed. The camera optical axis is a straight line originating from the optical center of the camera lens and perpendicular to the image sensor plane, representing the main optical direction of the camera's imaging system. The field of view refers to the range of spatial angles that the camera lens can observe, usually divided into horizontal and vertical field of view, which determines the size of the camera's observation range. The frustum represents the effective imaging area of ​​the camera in three-dimensional space. It is a frustum-shaped geometric body with the camera's optical center as its vertex, the front and rear sections as its boundaries, and the field of view as its subtended angle. The side boundaries refer to the planar boundaries that constitute the four sides of the frustum. These boundaries are determined by the camera's horizontal and vertical field of view, limiting the visible range of the calibration plate in the horizontal and vertical directions.

[0047] Specifically, after obtaining the near-end and far-end depth-of-field distances, the camera calibration system establishes a camera coordinate system with the camera's optical center as the origin. The Z-axis points forward and coincides with the optical axis, while the X and Y axes correspond to the horizontal and vertical directions of the image, respectively. Within this coordinate system, the camera calibration system defines the plane perpendicular to the Z-axis and at the near-end distance from the camera's optical center as the front section, and the plane perpendicular to the Z-axis and at the far-end distance as the rear section. These two sections constitute the boundary constraints of the frustum in the depth direction. The camera calibration system then determines the field of view. Specifically, although lens specifications typically provide a nominal field of view, this value is often based on a specific sensor size. To obtain more accurate geometric constraints, the camera calibration system calculates the precise horizontal and vertical field of view using the obtained camera hardware parameters (specifically, the horizontal physical dimensions W and vertical physical dimensions H of the image sensor) and lens hardware parameters (specifically, the focal length f). The calculation formula is as follows:

[0048] Horizontal field of view: =2×arc

[0049] Vertical field of view: FOVv = 2 × arc

[0050] The calculated angles are then converted to radians. Based on these angle parameters, the camera calibration system calculates the half-angle of the field of view and establishes the lateral constraint equations of the frustum using trigonometric relationships. The camera calibration system constructs planar equations for the left, right, upper, and lower sides, respectively. These planes all pass through the origin of the camera's optical axis and extend along the edge of the field of view. By combining the front and rear section constraints in the depth direction and the lateral boundary constraints in the four directions, the camera calibration system constructs a set of inequalities or plane normal vector equations describing the geometric characteristics of the frustum. Any point within the frustum satisfies all boundary constraints, thus ensuring that the pose of the subsequently generated calibration board is within the camera's clear imaging field of view, avoiding the generation of invalid or low-quality calibration data.

[0051] In some embodiments, the accurate construction of the view frustum can be achieved through a systematic geometric modeling method: the camera calibration system, in the camera coordinate system, provides a horizontal field of view (FOV). H And the vertical field of view (FOVv), convert it to radians:

[0052]

[0053] The camera calibration system defines the tangent of half the field of view as a boundary parameter:

[0054]

[0055] Then, the camera calibration system for any point The constraints located on the side of the view frustum are derived, with the left and right constraints derived as follows:

[0056]

[0057] The derivation of the upper and lower constraints is as follows:

[0058]

[0059] Next, the camera calibration system expresses the above constraints in the form of plane normals and dot products, with the defined normal vectors as follows: , Ultimately, the camera calibration system determines that point X within the optimal imaging frustum must simultaneously satisfy both the lateral normal constraint and the depth distance constraint:

[0060]

[0061] 103. The area enclosed by the front section, the rear section, and the side boundary is taken as the three-dimensional feasible space range of the calibration plate.

[0062] The three-dimensional feasible space range represents the set of valid locations for placing the calibration board. The calibration board images acquired within this range can be guaranteed to be both clear and focused and appear completely in the camera's field of view, meeting the data quality requirements of the calibration algorithm.

[0063] Specifically, the camera calibration system performs spatial Boolean operations or geometric intersection logic to combine the front section, rear section, and four side boundary planes. The system defines these six planes as constraints, dividing the three-dimensional space into an internal region that satisfies the constraints and an external region that does not. Specifically, this region is a truncated pyramid extending along the camera's optical axis, with its top surface being a rectangular region constrained by the front section, its bottom surface a rectangular region constrained by the rear section, and its side walls closed by four planes spanned by the field of view. The camera calibration system marks this closed region as the three-dimensional feasible space of the calibration board. All subsequent pose generation, sampling, and verification steps are strictly confined within this space to ensure that the generated calibration data is physically imageable and optically clear and valid.

[0064] Please see Figure 2 This is a schematic diagram of the optimal imaging cone in an embodiment of this application. Figure 2 The diagram illustrates the three-dimensional feasible space of the calibration board. This space is bounded by the camera's field of view boundary, as well as the near-end front section (light cyan surface) and far-end rear section (dark cyan surface) calculated based on depth of field. The blue shaded area represents the effective sampling space of the calibration board. Only when the calibration board is located inside this frustum geometry can it be guaranteed to be clearly imaged by the camera without exceeding the field of view.

[0065] 104. Sample the depth coordinates of the calibration board between the near-end distance and the far-end distance of the depth of field. Based on the depth coordinates and the hardware parameters of the calibration board, calculate the maximum allowable tilt angle of the calibration board normal vector relative to the camera optical axis.

[0066] The depth coordinate represents the position value of the calibration board's center point along the Z-axis of the camera coordinate system. It is the component of the distance from the calibration board to the camera's optical center projected onto the camera's optical axis. It is used to determine the specific position of the calibration board within the depth of field. For example, when the near end of the depth of field is 0.5 meters and the far end is 2.0 meters, the depth coordinate can be sampled between 0.5 meters and 2.0 meters. The calibration board normal vector represents the unit direction vector perpendicular to the calibration board plane and pointing towards the front of the calibration board. It is a mathematical expression used to describe the orientation of the calibration board in three-dimensional space. The angle between this vector and the camera's optical axis determines the degree of tilt of the calibration board. The camera optical axis represents the straight line emanating from the optical center of the camera lens and perpendicular to the image sensor plane. It refers to the main optical direction of the camera imaging system. The maximum allowable tilt angle represents the maximum allowable deflection angle of the calibration board normal vector relative to the camera's optical axis while ensuring image quality and feature recognizability.

[0067] Specifically, the camera calibration system selects several depth coordinate values ​​within a one-dimensional interval defined by the near-end distance and the far-end distance of the depth of field, either through random number generation or equal-interval division. Each depth coordinate value represents a candidate position of the calibration board's center point on the Z-axis of the camera coordinate system. For each sampled depth coordinate, the camera calibration system calculates the maximum allowable tilt angle of the calibration board at that depth position. First, the system calculates the diagonal length of the calibration board based on its width and height; this diagonal length represents the maximum geometric dimension of the calibration board in space. Next, the system calculates the distance difference between the current depth coordinate and the front and rear boundaries of the depth of field, selecting the smaller distance difference as the depth tolerance. This tolerance reflects the safety margin by which the calibration board can make depth shifts at its current position while maintaining sharp imaging. Finally, using trigonometric geometry, the system calculates the theoretical maximum tilt angle, limited by the depth of field range, based on the calibration board's diagonal length and the depth tolerance distance. Simultaneously, the camera calibration system also needs to consider the constraints on the camera imaging system's ability to effectively identify and extract corner features of the calibration board, and obtain the preset maximum feature-identifiable tilt angle. Ultimately, the camera calibration system selects the smaller value between the depth-of-field constraint tilt angle and the feature-identifiable tilt angle as the maximum permissible tilt angle of the calibration board normal vector relative to the camera optical axis at that depth coordinate position, ensuring that the generated calibration board pose guarantees both image sharpness and the detectability of corner features.

[0068] 105. Under the constraint of the maximum allowable tilt angle, generate the rotation matrix of the calibration plate, and sample the translation vector of the calibration plate within the field of view corresponding to the depth coordinate.

[0069] The rotation matrix represents the rotational transformation relationship of the calibration board coordinate system relative to the camera coordinate system, describing the orientation of the calibration board in three-dimensional space; the field of view section refers to the planar cross section of the view frustum perpendicular to the camera optical axis at a specific depth coordinate position. The shape of this section is usually rectangular, and its size is determined by the camera's field of view angle and the current depth distance; the translation vector is a vector containing three components (X, Y, Z), used to represent the three-dimensional spatial position of the calibration board's center point in the camera coordinate system.

[0070] Specifically, to generate the rotation matrix, the camera calibration system constructs a conical space centered on the camera's optical axis and with the maximum permissible tilt angle as the semi-apex angle. This conical space represents the range of all legal pointing directions of the calibration board's normal vector. The camera calibration system randomly samples a direction within this conical space as the principal normal direction of the calibration board, which determines the board's pitch and yaw attitude. Furthermore, since the calibration board's rotation around its own normal (i.e., roll angle) is generally not limited by depth of field, the camera calibration system also randomly generates a rotation angle. The camera calibration system combines the rotation transformation of the principal normal direction with the rotation transformation, generating the final calibration board rotation matrix through matrix multiplication. Subsequently, to determine the planar position of the calibration board, the camera calibration system calculates the field-of-view dimensions (i.e., the camera's field of view width and height) at that depth based on the current depth coordinates and the camera's field-of-view angle parameters. Within this field of view, the camera calibration system randomly samples the horizontal (X-axis) and vertical (Y-axis) coordinates of the calibration board's center. These two coordinates are then combined with the determined depth coordinates (Z-axis) to form the translation vector of the calibration board. At this point, the camera calibration system has generated complete pose data containing rotation and translation information.

[0071] 106. Construct a temporary pose based on the rotation matrix and translation vector, and calculate the corner coordinates of the calibration board in the temporary pose based on the hardware parameters of the calibration board.

[0072] Temporary pose refers to a data structure composed of the above rotation matrix and translation vector, which describes the spatial position and orientation of the calibration plate relative to the camera coordinate system at the current sampling time. It is usually represented as a rigid body transformation matrix. Corner coordinates refer to the specific spatial coordinates (X, Y, Z) of these feature points in the overall three-dimensional coordinate system of the camera after pose transformation.

[0073] Specifically, after generating the rotation matrix and translation vector of the calibration board through sampling and constraint calculations, the camera calibration system combines the two to construct the current temporary pose. To verify whether the calibration board is entirely within the effective imaging area in this pose, the camera calibration system first establishes a local coordinate system for the calibration board based on its hardware parameters (such as physical dimensions, number of rows and columns, spacing, etc.) and determines the original coordinates of all key feature points (corner points) in this local coordinate system. Specifically, to ensure the accuracy of subsequent calculations for the tilt angle constraints of the calibration board, the origin (0, 0, 0) of the local coordinate system of the calibration board is defined as the geometric center of the physical plane of the calibration board. Based on this definition, half the length of the diagonal of the calibration board calculated in the aforementioned steps is the geometric radius of the calibration board's rotation around the center. The coordinates of all corner points on the calibration board are symmetrically distributed coordinates defined relative to this geometric center. Next, the camera calibration system uses the principle of rigid body transformation to rotate these local corner point coordinates using a rotation matrix in the temporary pose, aligning their orientation with the camera coordinate system. Then, it superimposes translation vectors to move these points to their corresponding positions in the camera coordinate system. This process transforms the abstract pose parameters of the calibration board into the actual coordinate values ​​of each specific feature point on the calibration board in the camera's three-dimensional space.

[0074] In some embodiments, coordinate transformation calculations can be performed in the following manner: The camera calibration system sets the local corner points of the planar rectangular calibration plate as: (For example, the four vertices of the calibration board (top left, top right, bottom right, and bottom left) or all internal corner points), using the generated rotation matrix R and translation vector T, the corner points of the calibration board are transformed from the local coordinate system of the calibration board to the camera coordinate system through rigid body transformation. The specific mathematical processing logic is as follows: This yields a set of three-dimensional corner coordinates in the camera coordinate system. This is used for subsequent geometric verification.

[0075] 107. Remove temporary poses whose corner coordinates exceed the range of the three-dimensional feasible space, and at the same time remove temporary poses whose similarity to existing poses is higher than a preset threshold, to obtain a set of candidate poses.

[0076] Existing poses refer to historical pose data that have been verified and added to the candidate set during the current iteration generation process; similarity refers to the degree of closeness between the currently generated temporary pose and the existing pose in terms of spatial position distance and rotation angle difference; preset thresholds refer to numerical limits set by humans to determine whether two poses are duplicated or redundant, usually including translation distance thresholds and rotation angle thresholds; the candidate pose set refers to the list of valid calibration board pose data that has been retained after spatial validity verification and deduplication screening, and is prepared for final selection.

[0077] Specifically, after calculating the corner coordinates corresponding to the temporary pose, the camera calibration system performs a strict dual-screening logic. The first layer is geometric validity screening. The camera calibration system iterates through each calculated corner coordinate, determining whether it is completely contained within the three-dimensional feasible space (i.e., the frustum and depth of field). If any corner of the calibration board exceeds this range, it indicates that the pose will cause some of the calibration board image to be blurred or out of view, and the camera calibration system will directly discard the temporary pose. For poses that pass the first layer of screening, the camera calibration system further combines camera intrinsic parameters to project them onto the pixel plane, discarding those where the image exceeds the resolution range of the image sensor. The second layer is diversity screening. The camera calibration system calculates the similarity between the currently retained pose to be compared and every existing pose in the candidate pose set. The system first calculates the Euclidean distance of the translation position; if the distance is too close, it further calculates the relative rotation angle. If the difference between the two is below a preset threshold, it indicates that the current pose not only overlaps in position but also has the same posture, belonging to low-value redundant data, and the camera calibration system discards it. Only poses that simultaneously satisfy spatial constraints and possess sufficient diversity will be added to the candidate pose set by the camera calibration system.

[0078] In some embodiments, filtering and deduplication can be achieved as follows: The camera calibration system first performs frustum verification, substituting the corner points of the calibration plate into the hexahedral equation of the frustum obtained in the previous steps for judgment; if the conditions are not met, the frustum is discarded. Specifically, for any corner point X on the calibration plate... i It must simultaneously satisfy the front and rear section constraints and the side normal constraints of the view frustum. The specific mathematical verification conditions are as follows:

[0079]

[0080] Next, pixel range verification is performed, given that the camera's intrinsic parameter matrix includes focal length. , and the main point Using the projection formula, the three-dimensional points (x, y) in the camera coordinate system are projected. c y c , z c Converting coordinates (u, v) to pixel coordinates (u, v) is done using the following formula:

[0081]

[0082] Check if the obtained pixel coordinates exceed the image resolution W×H. If any corner point satisfies... If the pose fails to pass verification, it is discarded. For a verified pose (R, T) to be compared, the camera calibration system calculates its comparison with a pose in the existing pose set. Translational distance difference .like Less than the preset translation threshold (Minimum permissible translation difference), then further calculate the relative rotation matrix. And use the trace to calculate the relative rotation angle:

[0083]

[0084] If relative rotation angle Less than the preset rotation threshold If the minimum allowable rotation difference is found, the pose is determined to be too similar and is removed; otherwise, the pose is added to the candidate pose set.

[0085] 108. Calculate the translational distance metric and rotational distance metric between each candidate pose in the candidate pose set to obtain the comprehensive distance metric. Based on the comprehensive distance metric, iteratively select the candidate pose with the largest distance from the selected pose set and add it to the target pose set until the number of poses in the target pose set reaches the preset number.

[0086] Translational distance metric is used to represent the difference in Euclidean distance between the position coordinates of two poses in three-dimensional space; rotational distance metric is used to represent the difference in the geometric angle of the orientation of two poses in three-dimensional space; comprehensive distance metric is a single numerical index that can simultaneously reflect the differences in position and orientation after normalization and weighted summation; target pose set refers to the optimal subset of poses that maximizes the spatial distribution diversity and is ultimately used to guide the acquisition of calibration data; iterative selection is a greedy algorithm process based on the maximum-minimum distance strategy, used to select the pose that differs the most from the currently selected set.

[0087] Specifically, to ensure that the data used for calibration is as spatially uniform and diverse as possible, the camera calibration system selects a subset from the candidate pose set. The system first iterates through the candidate pose set, calculating the translational and rotational distances between any two poses. Since translational distance (usually in millimeters or meters) and rotational distance (usually in radians or degrees) belong to different physical units and have significantly different values, they cannot be directly compared. Therefore, the maximum translational distance and the theoretical maximum rotational angle in the entire set are obtained as normalization scales, mapping them to the same numerical range. Next, the camera calibration system, based on preset weighting coefficients (such as translational and rotational weights), merges the normalized translational and rotational distances into a comprehensive distance metric, thereby constructing a distance matrix describing the degree of difference between all poses. Based on this distance matrix, the camera calibration system performs iterative selection: First, a pose with the largest average distance to all other poses is selected as a seed point and added to the target pose set. Then, in each iteration, the distance from each remaining candidate pose to all selected poses in the target pose set is calculated, and the minimum value is taken as the "nearest distance" between the candidate pose and the selected set. Finally, the pose with the largest "nearest distance" among all candidate poses is selected and added to the target set. This process ensures that newly added poses are always located at the point of maximum "gap" in the current selected set, thus achieving optimal spatial coverage, until the number of selected poses reaches a preset requirement. The preset number is set based on the calibration algorithm's constraints on degrees of freedom and computational efficiency, and is generally recommended to be between 15 and 25. Too few poses will lead to unstable parameter solutions, while too many poses will increase the computational burden and have diminishing marginal returns on accuracy improvement.

[0088] In some embodiments, the calculation and selection of the optimal pose set can be achieved in the following manner: the camera calibration system sets the candidate pose set as follows: First, using the set of translation vectors for all poses {t i} Calculate the maximum translation difference between all poses, and use it as the distance translation normalization scale: .

[0089] For the rotational part, the geometric distance (SO(3) geodesic angle) between two arbitrary rotations is known to be:

[0090]

[0091] Since the two attitudes are furthest apart when θ = π, and are in the state of maximum separation of the rotation group, the rotation normalization scale is determined accordingly. = Next, the camera calibration system calculates the 6DoF normalized distance matrix pairwise for all poses and calculates the translational distance metric:

[0092]

[0093] To calculate the rotational distance metric, first calculate the relative rotation: and angle:

[0094]

[0095] get:

[0096] Subsequently, the camera calibration system calculates the integrated distance metric. ,in As the rotation distance weight, The translation distance weight is denoted as , where .

[0097] During the selection phase, the camera calibration system first calculates the average distance of each pose to all other poses:

[0098]

[0099] Extract the pose corresponding to the pose with the largest average distance as the initial pose of the target pose set. Let S be the index set of the selected poses. For each unselected pose, calculate its nearest distance to the selected set:

[0100]

[0101] choose The pose corresponding to the maximum value is added to the target pose set until the number of target poses in the set meets the requirements.

[0102] In some embodiments, after obtaining a target pose set containing rich position and attitude information through the above steps, the method provided in this application embodiment further includes: guiding calibration board data acquisition based on the target pose set. Taking visual guidance as an example, the camera calibration system first calculates the corner coordinates of the calibration board in the local coordinate system based on the acquired calibration board size (width and height), and uses these as template coordinates. Then, it iterates through the target pose set, and for each target pose in the set, it projects the template coordinates onto the camera's pixel coordinate system using the projection formula based on the acquired camera intrinsic parameters and the target pose (including rotation matrix and translation vector), obtaining the corresponding pixel projection points. Next, in the real-time video stream captured by the camera, each time the latest frame image is acquired, the calibration board template is transparently superimposed on the frame data based on the pixel projection points (e.g., drawing a semi-transparent virtual calibration board or corner connection frame), and the superposition result is returned to the UI visualization window for real-time display. In this way, the system intuitively guides the user to place the physical calibration board on the screen. When the user adjusts the calibration board to coincide with the transparent template displayed on the screen, the system can acquire valid image data under that pose.

[0103] The calibration board pose generation and selection method based on geometric constraints described in this application involves the camera calibration system first acquiring the hardware parameters of the camera, lens, and calibration board. The lens parameters are input into a thin lens model to obtain the near-far distance of the depth of field. A frustum is constructed based on the field of view, and the three-dimensional feasible space is determined, defining the effective area where the calibration board is clearly imaged and within the camera's field of view. Then, the camera calibration system samples depth coordinates within the depth of field, calculates the maximum allowable tilt angle, generates a rotation matrix and translation vector, constructs a temporary pose, and calculates the coordinates of the calibration board corner points. Next, the camera calibration system removes temporary poses whose corner coordinates exceed the three-dimensional feasible space or whose similarity to existing poses exceeds a threshold, obtaining a candidate pose set, thus alleviating the problem of invalid or redundant calibration board poses. Finally, the camera calibration system calculates the translation and rotation distance metrics of the candidate poses and obtains a comprehensive distance metric. The poses with the largest distance are iteratively selected to form the target pose set, solving the problems of convergent spatial positions and singular pose distribution in calibration, ultimately improving the accuracy of camera calibration.

[0104] The following provides a more detailed description of the process of the method provided in this implementation. Please refer to [link / reference]. Figure 3 This is another flowchart illustrating the calibration plate pose generation and screening method based on geometric constraints in this application embodiment.

[0105] 201. Obtain the hardware parameters of the camera, lens, and calibration board respectively. Input the lens hardware parameters into the thin lens model to obtain the near-field distance and far-field distance of the camera at the preset focus distance. (Refer to the execution process of step 101)

[0106] 202. Determine the plane located at the near end of the depth of field and perpendicular to the camera's optical axis as the front section, and the plane located at the far end of the depth of field and perpendicular to the camera's optical axis as the rear section. Determine the field of view angle based on the camera's hardware parameters, and construct the side boundary of the field of view cone based on the field of view angle. (Refer to the execution process of step 102)

[0107] 203. The area enclosed by the front section, the rear section, and the side boundary is defined as the three-dimensional feasible space of the calibration plate. (Refer to the execution process of step 103)

[0108] 204. Sample the depth coordinates of the calibration board between the near-end distance and the far-end distance of the depth of field. (Refer to the execution process of step 104)

[0109] 205. Calculate the diagonal length of the calibration plate based on its width and height.

[0110] The diagonal length represents the straight-line distance between two opposite vertices on the calibration plate, and is used to represent the maximum geometric span required for the calibration plate to rotate in space.

[0111] Specifically, the camera calibration system reads the pre-stored or user-inputted calibration board width *w* and height *h*. To accurately assess the impact of the calibration board's rotation on depth of field in three-dimensional space, the system uses the Pythagorean theorem to calculate the length of the calibration board's geometric diagonal. This length serves as the reference data for subsequently calculating the rotation radius. The camera calibration system calculates the calibration board diagonal length *L*: Through this calculation, the camera calibration system determined the farthest boundary distance that the calibration plate might reach during rotation, providing a geometric basis for subsequent tilt angle constraint calculations.

[0112] 206. Calculate the first distance difference between the depth coordinates and the near end of the depth of field, calculate the second distance difference between the depth coordinates and the far end of the depth of field, and select the minimum value between the first distance difference and the second distance difference as the depth tolerance distance.

[0113] The first distance difference (d1) and the second distance difference (d2) represent the spatial distances from the current sampling position to the front and rear depth boundaries, respectively; the depth tolerance distance refers to the safety margin that the calibration board is allowed to move in the depth direction at the current depth position without exceeding the range of clear imaging.

[0114] Specifically, the camera calibration system performs this step after sampling a depth coordinate within the three-dimensional feasible space. First, the camera calibration system acquires the near-field distance and far-field distance previously calculated using the thin lens model. Next, the system calculates the absolute difference between the depth coordinate and the near-field distance, obtaining the first distance difference; simultaneously, it calculates the absolute difference between the depth coordinate and the far-field distance, obtaining the second distance difference. These two distance differences represent the maximum usable space when the calibration board is tilted forward and backward, respectively. To ensure that the calibration board does not exceed the depth of field regardless of which direction it is tilted, the camera calibration system compares the values ​​of these two distance differences and selects the minimum value as the depth tolerance distance. For example, if the calibration board is 1 meter from the near end and 2 meters from the far end, the camera calibration system selects 1 meter as the depth tolerance distance at that position to ensure the most stringent constraints.

[0115] 207. Based on the diagonal length and depth tolerance distance, calculate the first tilt angle limited by the depth of field using trigonometric relationships. The depth of field range is determined based on the near-end distance and the far-end distance.

[0116] The first tilt angle represents the maximum allowable deflection angle of the calibration board plane normal vector relative to the camera optical axis when only considering the optical imaging sharpness constraint. It is a physical constraint index limited by the depth of field range. The trigonometric relationship refers to the spatial geometric mapping relationship described by trigonometric functions such as the sine function.

[0117] Specifically, in order to ensure that all corner points of the calibration board remain within the depth of field after tilting, the camera calibration system needs to calculate the first restricted tilt angle. The calculation process consists of two steps: First, the camera calibration system calculates the sinusoidal limit value based on the ratio between the depth tolerance distance (i.e., the minimum of d1 and d2) and the rotation radius of the calibration plate (half of the diagonal length L). : ,in, This represents the depth tolerance distance. Then, the camera calibration system uses the arcsine function to apply the above calculations... The value is converted into an angle value to obtain the first tilt angle constrained by the depth of field. : .

[0118] 208. Obtain the preset maximum recognizable tilt angle allowed by the camera for imaging, and use it as the second tilt angle. Determine the minimum value between the first tilt angle and the second tilt angle as the maximum allowable tilt angle of the calibration plate normal vector relative to the camera optical axis.

[0119] The preset maximum identifiable tilt angle represents the limit angle at which the calibration algorithm can effectively extract corner features when processing images. It is an empirical threshold determined by the performance of the feature extraction algorithm and is defined as the second tilt angle (MAXTILT). The maximum allowable tilt angle of the calibration board normal vector relative to the camera optical axis represents the upper limit of the comprehensive angle used to constrain pose generation. It is used to represent the angle range that simultaneously meets the dual conditions of image sharpness and algorithm identifiability.

[0120] Specifically, the camera calibration system needs to consider not only the physical limitations imposed by depth of field but also the limitations of the algorithmic recognition. The camera calibration system reads the pre-set MAXTILT parameters. Subsequently, the camera calibration system uses the depth-limited parameters calculated in step 207... The smaller of the two values ​​is selected as the final maximum allowable tilt angle, compared to the algorithm's limit MAXTILT, ensuring that this angle is non-negative. Maximum allowable tilt angle: The final decision This ensures that the calibration board is both within a clear imaging range and within an angle range where feature points can be accurately extracted.

[0121] 209. Under the constraint of the maximum allowable tilt angle, generate the rotation matrix of the calibration plate, and sample the translation vector of the calibration plate within the field of view corresponding to the depth coordinate.

[0122] The steps include: constructing a conical space with the camera optical axis as the central axis and the maximum allowable tilt angle as the semi-vertical angle; randomly sampling the direction angle and tilt angle of the calibration plate normal vector within the conical space, and determining the intermediate normal vector based on the direction angle and tilt angle; based on the intermediate normal vector and the direction vector of the camera optical axis, using the principle of axis-angle rotation transformation, constructing a first rotation matrix that rotates the camera optical axis to coincide with the intermediate normal vector; randomly sampling the rotation angle of the calibration plate around its own normal, and constructing a second rotation matrix based on the rotation angle; multiplying the first rotation matrix and the second rotation matrix to obtain the rotation matrix.

[0123] Please see Figure 4 This is a schematic diagram of a vector-constrained conical space in an embodiment of this application. To generate a reasonable rotation matrix, a conical space centered on the camera's optical axis is constructed. As shown in the figure, ∠θ represents the calculated maximum allowable tilt angle (half-apex angle). The normal to the calibration plate (i.e., the calibration plate normal vector) is randomly sampled within this conical space. This geometric constraint ensures that the tilt of the calibration plate surface relative to the camera is always within an angle range that guarantees both clear imaging and effective recognition by the algorithm.

[0124] Here, the conical space represents a three-dimensional geometric region centered on the camera's optical axis and with the maximum allowable tilt angle as its semi-vertical angle, used to limit the pointing range of the calibration plate's normal vector; the orientation angle (φ) and tilt angle (θ) are two angular parameters describing the pointing of the normal vector in spherical coordinates; the first rotation matrix (R... dir The first rotation matrix (R) represents the transformation matrix that rotates the camera's optical axis to the direction of the sampled intermediate normal vector; the second rotation matrix (R) represents the rotation angle of the calibration plate around its own normal axis, used to increase pose diversity; the third rotation matrix (R) represents the rotation angle of the calibration plate around its own normal axis. roll ) represents the rotation transformation matrix generated based on the rotation angle; the field of view section refers to the rectangular visible area determined by the horizontal and vertical field of view angles of the camera at a specific depth distance.

[0125] Specifically, the camera calibration system determines the maximum permissible tilt angle. and depth coordinates b z Then, pose data is generated in two parts.

[0126] First, the camera calibration system generates a rotation matrix. The camera calibration system knows that all valid normal vectors must lie on a point centered on the Z-axis with an opening angle of . Inside the conical surface. The camera calibration system is in the interval... Randomly sample the tilt angle θ within the interval The internal random sampling direction angle φ is used. The camera calibration system calculates the calibration plate normal vector using the spherical coordinate rotation direction vector theorem. :

[0127]

[0128] Next, the camera calibration system is based on the camera optical axis direction vector. and the calculated intermediate normal vector Construct a first rotation matrix Rdir that rotates the Z-axis to the normal direction. When θ≈0 Otherwise, the camera calibration system calculates the rotation axis vector a:

[0129]

[0130]

[0131] The camera calibration system determines the rotation vector based on the Rodriguez formula. : And construct an antisymmetric matrix :

[0132]

[0133] This leads to the first rotation matrix: Since the rotation of the calibration plate around its own normal is not limited by the depth of field, the camera calibration system randomly samples the rotation angle ρ within the range of U(0,2π) to construct the rotation vector. And calculate the second rotation matrix. : Finally, the rotation matrix is ​​obtained. : .

[0134] After obtaining the rotational attitude of the calibration board within the frustum, the field of view size at the current bz distance is calculated based on the lens FOV, and then the center of the calibration board is randomly sampled within this range. as well as The coordinates are used to obtain the complete center coordinates of the calibration board, which is the translation vector of the calibration board relative to the camera.

[0135] Given the horizontal field of view of the camera and vertical field of view At the current distance, the camera's field of view is wide and high. for:

[0136]

[0137]

[0138] Transform to camera coordinate system and perform and If the coordinates are randomly sampled, then:

[0139]

[0140]

[0141] The translation vector of the calibration board in the camera coordinate system is obtained by integration. :

[0142]

[0143] 210. Construct a temporary pose based on the rotation matrix and translation vector, and calculate the corner coordinates of the calibration board in the temporary pose based on the hardware parameters of the calibration board. (Refer to the execution process of step 106)

[0144] 211. Remove temporary poses whose corner coordinates exceed the 3D feasible space range, and also remove temporary poses whose similarity to existing poses is higher than a preset threshold, to obtain a candidate pose set. (Refer to the execution process of step 107)

[0145] 212. Calculate the translational distance metric and rotational distance metric between each candidate pose in the candidate pose set to obtain the comprehensive distance metric. Using the comprehensive distance metric as a benchmark, iteratively select the candidate pose with the largest distance from the selected pose set and add it to the target pose set until the number of poses in the target pose set reaches the preset number. (Refer to the execution process of step 108)

[0146] The calibration board pose generation and selection method based on geometric constraints described in this application involves the camera calibration system first acquiring the hardware parameters of the camera, lens, and calibration board. It then calculates the near-far distance of the depth of field using a thin lens model, constructs a viewing frustum based on the field of view, and determines the three-dimensional feasible space range. Next, the camera calibration system samples depth coordinates, calculates the diagonal length and depth tolerance distance based on the calibration board hardware parameters, and determines the first tilt angle constrained by the depth of field and a preset second tilt angle. The minimum value is taken to obtain the maximum allowable tilt angle, balancing clear depth-of-field imaging with feature recognizability, thus providing a reasonable constraint for rotation matrix generation. Then, under the constraint of the maximum allowable tilt angle, the camera calibration system generates a rotation matrix and translation vector, constructs a temporary pose, and calculates corner coordinates. Temporary poses with corners exceeding the feasible space and similarity exceeding the threshold are eliminated to obtain a candidate pose set, alleviating the problems of invalid and redundant poses. Finally, the camera calibration system calculates the comprehensive distance metric of the candidate poses and iteratively selects the poses with the largest distance to form the target pose set. This alleviates the problem of the poses in the calibration space being similar and the pose distribution being singular, allowing the poses to cover the camera's field of view and depth range, and ultimately improving the accuracy of camera calibration.

[0147] The methods provided in the above embodiments can be executed by a camera calibration system. The camera calibration system in the embodiments of this invention is described below from a hardware processing perspective; please refer to [link to relevant documentation]. Figure 5 This is a schematic diagram of the physical device structure of a camera calibration system in an embodiment of this application.

[0148] It should be noted that, Figure 5 The structure of the camera calibration system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0149] like Figure 5As shown, the camera calibration system includes a processing unit (CPU+GPU) 301, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 302 or a program loaded from storage section 308 into random access memory (RAM) 303, such as performing the methods described in the above embodiments. The RAM 303 also stores various programs and data required for system operation. The CPU+GPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0150] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including a hard disk, etc.; and communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0151] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by the central processing unit (CPU+GPU) 301, it performs the various functions defined in the present invention.

[0152] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0154] Specifically, the camera calibration system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the calibration board pose generation and screening method based on geometric constraints provided in the above embodiment.

[0155] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the camera calibration system described in the above embodiments; or it may exist independently and not assembled into the camera calibration system. The storage medium carries one or more computer programs that, when executed by a processor of the camera calibration system, cause the camera calibration system to implement the geometric constraint-based calibration board pose generation and selection method provided in the above embodiments.

[0156] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0157] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0158] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for generating and filtering the pose of a calibration plate based on geometric constraints, characterized in that, Applied to a camera calibration system, the camera calibration system including a camera, a lens, and a calibration plate, the method includes: The hardware parameters of the camera, the lens, and the calibration plate are obtained respectively. The hardware parameters of the lens are input into the thin lens model to obtain the near-field distance and the far-field distance of the camera at the preset focusing distance. The plane located at the near end of the depth of field and perpendicular to the camera optical axis is defined as the front section, and the plane located at the far end of the depth of field and perpendicular to the camera optical axis is defined as the rear section. The field of view is determined according to the camera's hardware parameters, and the side boundary of the view frustum is constructed based on the field of view. The area enclosed by the front section, the rear section, and the side boundary is defined as the three-dimensional feasible space range of the calibration plate; The depth coordinates of the calibration board are sampled between the near-end distance and the far-end distance of the depth of field. Based on the depth coordinates and the hardware parameters of the calibration board, the maximum allowable tilt angle of the calibration board normal vector relative to the camera optical axis is calculated. Under the constraint of the maximum allowable tilt angle, the rotation matrix of the calibration plate is generated, and the translation vector of the calibration plate is sampled within the field of view corresponding to the depth coordinate. A temporary pose is constructed based on the rotation matrix and the translation vector, and the corner coordinates of the calibration board in the temporary pose are calculated based on the hardware parameters of the calibration board. Temporary poses whose corner coordinates exceed the range of the three-dimensional feasible space are removed, and temporary poses whose similarity to existing poses is higher than a preset threshold are also removed, to obtain a set of candidate poses. The translational distance metric and rotational distance metric between each candidate pose in the candidate pose set are calculated respectively to obtain a comprehensive distance metric. Based on the comprehensive distance metric, the candidate pose with the largest distance from the selected pose set is iteratively selected and added to the target pose set until the number of poses in the target pose set reaches a preset number.

2. The method according to claim 1, characterized in that, The step of calculating the translational distance metric and the rotational distance metric between each candidate pose in the candidate pose set to obtain a comprehensive distance metric specifically includes: Calculate the translational distance metric and the rotational distance metric between each candidate pose in the candidate pose set; Obtain the translation distance between any two poses in the candidate pose set, and determine the translation distance with the largest value as the translation normalization scale, and determine the theoretical maximum allowable relative rotation angle between any two poses as the rotation normalization scale; The translation distance metric is normalized using the translation normalization scale, and the rotation distance metric is normalized using the rotation normalization scale. Based on preset weighting coefficients, the normalized translation distance metric and the normalized rotation distance metric are weighted and summed to obtain the comprehensive distance metric.

3. The method according to claim 1 or 2, characterized in that, Based on the comprehensive distance metric, the candidate pose with the largest distance to the selected pose set is iteratively selected and added to the target pose set until the number of poses in the target pose set reaches a preset number, specifically including: Based on the comprehensive distance metric, the average distance between each candidate pose in the candidate pose set and all other candidate poses is calculated; Select the candidate pose with the largest average distance as the initial pose, and add the initial pose to the target pose set; For each remaining candidate pose in the candidate pose set, calculate the comprehensive distance metric between each candidate pose and all poses in the target pose set, and select the minimum value of the comprehensive distance metric as the closest distance between the candidate pose and the target pose set. Compare the nearest distances corresponding to all the candidate poses, and select the candidate pose corresponding to the nearest distance with the largest value to add it to the target pose set; Repeat the process of calculating the nearest distance and selecting the largest value until the number of poses in the target pose set reaches a preset number.

4. The method according to claim 1, characterized in that, Based on the depth coordinates and the hardware parameters of the calibration board, the maximum allowable tilt angle of the calibration board normal vector relative to the camera optical axis is calculated, specifically including: Calculate the diagonal length of the calibration plate based on its width and height; Calculate the first distance difference between the depth coordinates and the near-end distance of the depth of field, calculate the second distance difference between the depth coordinates and the far-end distance of the depth of field, and select the minimum value between the first distance difference and the second distance difference as the depth tolerance distance; Based on the diagonal length and the depth tolerance distance, a first tilt angle limited to the depth of field range is calculated using trigonometric geometry, wherein the depth of field range is determined based on the near-end distance and the far-end distance of the depth of field. The preset maximum recognizable tilt angle allowed by the camera for imaging is obtained as the second tilt angle. The minimum value between the first tilt angle and the second tilt angle is determined as the maximum allowable tilt angle of the calibration plate normal vector relative to the camera optical axis.

5. The method according to claim 1 or 4, characterized in that, Under the constraint of the maximum allowable tilt angle, the rotation matrix of the calibration plate is generated, specifically including: Construct a conical space with the camera optical axis as the central axis and the maximum allowable tilt angle as the half-apex angle. Randomly sample the direction angle and tilt angle of the calibration plate normal vector within the conical space, and determine the intermediate normal vector based on the direction angle and the tilt angle. Based on the intermediate normal vector and the direction vector of the camera optical axis, a first rotation matrix is ​​constructed to rotate the camera optical axis to coincide with the intermediate normal vector using the principle of axis-angle rotation transformation. The rotation angle of the calibration plate around its own normal is randomly sampled, and a second rotation matrix is ​​constructed based on the rotation angle; Multiply the first rotation matrix by the second rotation matrix to obtain the rotation matrix.

6. The method according to claim 1, characterized in that, Temporary poses whose corner coordinates exceed the range of the three-dimensional feasible space are removed, and temporary poses with a similarity to existing poses higher than a preset threshold are also removed, resulting in a candidate pose set, specifically including: Temporary poses whose corner coordinates exceed the range of the three-dimensional feasible space are removed, and the corner coordinates of the retained temporary poses are projected onto the pixel plane according to the camera's hardware parameters to obtain pixel coordinates; Temporary poses where the pixel coordinates are outside the imaging range of the image sensor are removed to obtain the poses to be compared. Calculate the translation distance difference between the pose to be compared and the existing pose; when the translation distance difference is less than a preset translation threshold, calculate the relative rotation angle between the pose to be compared and the existing pose. The poses to be compared that have a relative rotation angle less than a preset rotation threshold are removed, and the poses to be compared that have not been removed are added to the candidate pose set.

7. The method according to claim 1, characterized in that, By inputting the hardware parameters of the lens into a thin lens model, the near-field distance and far-field distance of the camera at a preset focusing distance are obtained, specifically including: Input the hardware parameters of the lens into the thin lens model; The hyperfocal distance of the lens is calculated based on its focal length, aperture number, and permissible circle of confusion diameter. The hyperfocal distance is directly proportional to the square of the focal length and inversely proportional to the product of the aperture number and the permissible circle of confusion diameter. Based on the hyperfocal distance and the preset focus distance, the near-end distance and the far-end distance of the depth of field are calculated.

8. A camera calibration system, characterized in that, Includes camera, lens, and calibration plate; The camera calibration system includes one or more processors and a memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the camera calibration system to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising computer instructions, characterized in that, When the computer instructions are executed on the camera calibration system, the camera calibration system performs the method as described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are run on the camera calibration system, the camera calibration system performs the method as described in any one of claims 1-7.

Images