A high-precision automatic calibration method for multiple cameras based on machine learning
By combining machine learning and Wiberg calibration targets with relative motion error bound constraints, the problems of inconsistent reliability of observation constraints and pose ambiguity in multi-camera calibration were solved, achieving high-precision and stable calibration results.
Patent Information
- Application Number
- CN202611490639.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-23
- Publication Date
- 2026-11-17
AI Technical Summary
Existing multi-camera calibration methods struggle to accurately reflect the reliability of observation constraints under conditions of image blurring, occlusion, and perspective changes, leading to inconsistent calibration parameters. Furthermore, local pose solutions can easily mask ambiguities in planar calibration, resulting in unstable inner-layer pose responses. Simply increasing the number of images makes it difficult to distinguish competing branches, thus affecting calibration accuracy and stability.
The machine learning observation model generates positioning uncertainty, and combined with Wiberg calibration target and relative motion error bound constraints, it performs inner-layer pose solving and pseudo-arc length extension to reduce pose ambiguity. By selecting supplementary motion through bidirectional refitting, the calibration accuracy and stability are improved.
It improves the accuracy and stability of multi-camera calibration, reduces pose ambiguity and erroneous updates, enhances the ability to distinguish competing branches, and ensures the reliability and consistency of calibration results.
Smart Images

Figure CN123118787A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of camera calibration technology, and in particular to a high-precision automatic calibration method for multiple cameras based on machine learning. Background Technology
[0002] Multi-camera calibration is used to determine the imaging parameters and spatial positional relationships of cameras, and is the foundation of 3D measurement, industrial vision inspection, and spatial positioning. Calibration deviations between cameras can affect cross-view feature matching and spatial coordinate reconstruction; therefore, it is necessary to obtain consistent calibration parameters under a unified coordinate system and spatial scale.
[0003] Existing calibration methods typically acquire images of a calibration board of known size at different positions and orientations, establish the projection relationship between image feature points and the physical coordinates of the calibration board, and obtain calibration parameters through parameter initialization and reprojection error optimization. Machine learning models can handle feature point localization and localization uncertainty estimation, while hierarchical optimization methods can divide camera parameters and calibration board pose into interrelated solution variables.
[0004] However, image blurring, occlusion, and perspective changes can cause inconsistencies in observation and localization quality, making it difficult to accurately reflect the reliability of observation constraints using uniform weights. Planar calibration boards exhibit pose ambiguity under certain observation conditions, with different poses potentially having similar reprojection errors. For local optimization methods that retain only a single inner-layer pose solution, a decrease in the target value does not necessarily indicate that the calibration parameters have eliminated ambiguity.
[0005] Furthermore, the response of inner-layer pose to changes in camera parameters may be unstable near degenerate positions, and the lack of branch continuity verification can easily cause solution jumps. Simply increasing the number of images may not necessarily enhance the ability to distinguish between competing solutions, and independently adjusting the pose of each acquisition may also absorb some camera parameter deviations. Therefore, it is still necessary to balance observation reliability, competitive solution identification, and motion constraints between acquisitions to improve the stability of automatic calibration results.
[0006] Therefore, how to provide a high-precision automatic calibration method for multiple cameras based on machine learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0007] One objective of this invention is to propose a high-precision automatic calibration method for multiple cameras based on machine learning. This invention identifies and tracks competing calibration branches by eliminating known roots and using pseudo-arc length extension, and selects supplementary motion by bidirectional refitting under relative motion error bound constraints. It integrates pairwise observation constraints into the inner-layer pose solution, reduces pose ambiguity and erroneous branch updates, and improves the accuracy and stability of automatic calibration for multiple cameras.
[0008] A high-precision automatic calibration method for multiple cameras based on machine learning according to an embodiment of the present invention includes the following steps: Acquire synchronized images of a calibration board of known size from multiple cameras and the relative motion error boundary, establish a unified calibration benchmark, and generate calibration basis data; The calibration baseline data is input into the machine learning observation model to generate a calibration observation set containing positioning uncertainty; Establish a Wiberg calibration target using the calibration observation set, perform inner layer pose solving, and generate the initial calibration state; Read the initial calibration state, perform known root elimination on the inner stationary point equation, determine the observation error boundary according to the positioning uncertainty, verify the pose according to the Wiberg calibration target, and form a competitive pose set; For the competitive pose set, pseudo-arc length extension is performed on the inner pose and path advance, the outer update is checked according to the sub-support connection relationship, competitive branches are screened according to the observation error boundary and calibration tolerance, and a competitive calibration branch set is generated. Perform bidirectional refitting under relative motion error bound constraints on the competitive calibration branch set, select the pair of motion reshoots with the smallest remaining observation difference, update the calibration observation set, associate the pair of inner layer poses according to the relative motion error bound and reorganize the inner layer variables, update the initial calibration state and iterate to form the calibration iteration state; Check the calibration iteration status, and output the multi-camera calibration parameters under the unified calibration benchmark after verification of calibration completion.
[0009] Optionally, obtaining the calibration baseline data specifically includes: Acquire synchronized images of a calibration board of known size from multiple cameras, establish image associations according to the synchronized acquisition batches, and generate synchronized acquisition data; Based on the independent verification records of the calibration plate motion mechanism matched with the synchronously acquired data, the relative motion error boundary is extracted and associated with the synchronously acquired data to generate motion constraint acquisition data; A reference camera is selected based on the motion constraint acquisition data, a unified calibration benchmark is established and associated with the motion constraint acquisition data, and calibration basis data is generated.
[0010] Optionally, obtaining the calibration observation set containing positioning uncertainty specifically includes: The calibration base data is input into the machine learning observation model, and feature encoding is performed on the synchronized images while retaining the acquisition correlation to generate an observation feature set. The observed feature set is fed into the location prediction branch and the uncertainty prediction branch to determine the location of the calibration feature points and estimate the positioning uncertainty, thereby generating positioning observation data. Based on the positioning observation data and the geometric information of the calibration board, calibration feature points are matched and then generated according to the collected and associated matching results, resulting in a calibration observation set containing positioning uncertainty.
[0011] Optionally, obtaining the initial calibration state specifically includes: Read the calibration observation set, configure the camera parameters and inner layer pose according to the unified calibration benchmark, establish parameter sharing relationship across acquisition batches and perform initialization to form the calibration parameter configuration; The projection positions of calibration feature points are calculated based on the calibration parameters. The reprojection deviation is constructed using the projection positions and the positions of calibration feature points in the calibration observation set. The reprojection deviation is weighted and summarized according to the positioning uncertainty to establish the Wiberg calibration target. For Wiberg calibration target with fixed camera parameters, an inner layer pose problem is constructed according to synchronous acquisition batches. Common observation constraints for sharing the inner layer pose are collected, the inner layer pose is iteratively updated, and the inner layer solution state is generated. Based on the inner layer solution state, establish the inner layer stationary point equation, differentiate with respect to the inner layer pose and camera parameters respectively, construct the inner layer pose response equation and record the response solution state, and generate the initial calibration state by associating the inner layer solution state.
[0012] Optionally, obtaining the competitive pose set specifically includes: Read the initial calibration state, extract the inner layer stationary point equations and the obtained pose solutions, establish known root records according to the synchronous acquisition batches, construct the inner layer stationary point equations after excluding known roots, and form the excluded solution state. Based on the exclusion solution state, perform inner stationary point search, substitute the search results back into the inner stationary point equation without the exclusion factor for stationary point verification, add the new pose solution that passes the stationary point verification to the known root record and update the exclusion solution state, and gather the obtained pose solutions and the new pose solutions to generate a candidate stationary point set. For the candidate stationary point set, the observation error boundary is determined according to the positioning uncertainty. The candidate stationary points are substituted back into the Wiberg calibration target, the reprojection error is calculated, and the local minima verification and the physical feasibility verification of the common observation are performed to generate the pose verification state. Read the pose verification status, retain the poses that pass the local minima verification and the common observation physical feasibility verification and satisfy the observation error bound, perform pose deduplication, reconstruct the inner pose response equation for the retained poses and record the response solution status to form a competitive pose set.
[0013] Optionally, obtaining the competitive calibration branch set specifically includes: Read the competitive pose set, organize the calibration branch according to the synchronous acquisition batch and camera parameters, determine the camera parameter trial direction and configure the path advance, establish the joint extension relationship between the inner pose and the path advance, and generate the extension initial state; For the initial state of the extension, calculate the tangential direction of the joint extension relationship, predict the inner layer pose and path advance along the tangential direction, perform joint correction by combining the inner layer stationary point equation and pseudo-arc length constraint, record the correction results and the sub-support connection relationship, and generate the pose extension state. Candidate camera parameters are recovered based on pose extension state, inner layer pose is verified according to branch connection relationship, outer layer update is checked under Wiberg calibration target, and the verified camera parameters are associated with inner layer pose to generate calibration branch set; Read the calibration branch set, calculate the reprojection error of each calibration branch using the same calibration observation set, retain the effective branches according to the observation error boundary, compare the differences in camera parameters of the effective branches according to the calibration tolerance, form competitive branch pairs with out-of-tolerance terms, and generate a competitive calibration branch set.
[0014] Optionally, obtaining the calibration iteration state specifically includes: Read the competitive calibration branch set, generate candidate pairs of motion within the common visible range and the reachable range of the motion mechanism, fix the camera parameters of the competitive branch pairs, and call the Wiberg inner layer solver under the relative motion error bound to perform bidirectional refitting and generate bidirectional refitting results. Based on the bidirectional refit results, the minimum bidirectional residual observation difference between the competing branch pairs of each candidate pair motion is taken, and the candidate pair motion with the largest minimum value is selected. The calibration plate motion mechanism is controlled to execute the pair motion and trigger synchronous acquisition to generate a set of supplementary images. The re-captured image set is input into the machine learning observation model to update the calibration observation set. The inner layer poses are associated according to the relative motion error boundary. The inner layer variables of the pair acquisition group are reconstructed by sharing the initial pose and the constrained relative motion, and the Wiberg calibration target is reconstructed to form the associated calibration state. Read the associated calibration state, re-execute the inner layer pose solution and update the initial calibration state, repeat the known root elimination, pseudo-arc length extension and competitive branch determination, and perform bidirectional refitting and paired motion supplementation on the retained competitive branches to form the calibration iteration state.
[0015] Optionally, obtaining the multi-camera calibration parameters under the unified calibration benchmark specifically includes: Read the calibration iteration status, check the divergence judgment results, verify the reprojection error of the retained calibration branch according to the observation error boundary, and extract the calibration parameters to be verified from the calibration branch that has passed the verification and the divergence has been eliminated. Fix the calibration parameters to be verified, read the synchronous image of the known size calibration board that has not participated in the calibration solution, match the cross-camera calibration feature points and reconstruct the spatial coordinates, compare the distance between the reconstructed points with the distance between the points in the geometric information of the calibration board, and generate the calibration completion verification result. Read the calibration completion verification results. When the verification is successful, organize the calibration parameters to be verified according to the camera identification, associate them with the unified calibration benchmark, and output the calibration parameters of multiple cameras.
[0016] The beneficial effects of this invention are: This invention addresses the issue of inconsistent positioning quality of calibration feature points observed by different cameras. It incorporates the positioning uncertainty generated by the machine learning observation model into the Wiberg calibration objective and establishes cross-camera common observation constraints by sharing the inner layer pose in the same synchronous acquisition batch. This makes the impact of observation on calibration parameters commensurate with the positioning reliability, reducing the interference of low-reliability observations on parameter estimation. At the same time, the inner layer pose response reflects the correlation between camera parameters and calibration board pose, improving the geometric consistency of multi-camera calibration solutions.
[0017] This invention addresses the problem that a single local pose solution can easily obscure ambiguities in planar calibration. It introduces the exclusion of known roots into the inner stationary point equation, reducing the repeated return to known poses during the search. Furthermore, it utilizes the Wiberg calibration target without an exclusion factor to verify the search results, ensuring that competing poses are preserved and compared under a unified evaluation scale. This avoids directly treating a single convergence result as a definite calibration result and reduces the risk of missing poses with similar reprojection errors but different spatial interpretations.
[0018] This invention addresses the issues of degradation in inner-layer pose response and branch jumps during outer-layer updates. It incorporates inner-layer pose and path advance into pseudo-arc length extension and verifies outer-layer updates through branch-support connection relationships. This ensures that solution branches satisfying the extension conditions maintain continuous tracking near turning points, reducing erroneous updates caused by local response instability. Furthermore, it combines observation error bounds and calibration tolerances to distinguish whether observation fitting is effective and whether camera parameters still have discrepancies, thereby improving the reliability of competitive calibration branch selection.
[0019] This invention addresses the problem that simply increasing the number of images is insufficient to effectively distinguish competing branches. It evaluates the value of reshoots by using the remaining observation differences after bidirectional refitting under relative motion error bounds. This allows the selection of reshoots to consider the ability of competing branches to absorb observation differences through pose adjustment, enhancing the discriminative effect of new observations. Furthermore, it incorporates the motion correlation of paired acquisitions into the inner-layer variables, limiting the compensation of camera parameter deviations by independent pose adjustments between the two acquisitions. This allows reshoot information to directly contribute to the Wiberg calibration solution, improving the accuracy and stability of automatic calibration for multiple cameras. Attached Figure Description
[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1This is a flowchart of a high-precision automatic calibration method for multiple cameras based on machine learning proposed in this invention; Figure 2 This is a schematic diagram of the pairwise inner layer pose association of a high-precision automatic calibration method for multiple cameras based on machine learning proposed in this invention. Detailed Implementation
[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0022] refer to Figures 1-2 A high-precision automatic calibration method for multiple cameras based on machine learning includes the following steps: Acquire synchronized images of a calibration board of known size from multiple cameras and the relative motion error boundary, establish a unified calibration benchmark, and generate calibration basis data; The calibration baseline data is input into the machine learning observation model to generate a calibration observation set containing positioning uncertainty; Establish a Wiberg calibration target using the calibration observation set, perform inner layer pose solving, and generate the initial calibration state; Read the initial calibration state, perform known root elimination on the inner stationary point equation, determine the observation error boundary according to the positioning uncertainty, verify the pose according to the Wiberg calibration target, and form a competitive pose set; For the competitive pose set, pseudo-arc length extension is performed on the inner pose and path advance, the outer update is checked according to the sub-support connection relationship, competitive branches are screened according to the observation error boundary and calibration tolerance, and a competitive calibration branch set is generated. Perform bidirectional refitting under relative motion error bound constraints on the competitive calibration branch set, select the pair of motion reshoots with the smallest remaining observation difference, update the calibration observation set, associate the pair of inner layer poses according to the relative motion error bound and reorganize the inner layer variables, update the initial calibration state and iterate to form the calibration iteration state; Check the calibration iteration status, and output the multi-camera calibration parameters under the unified calibration benchmark after verification of calibration completion.
[0023] In this embodiment, obtaining the calibration baseline data specifically includes: Acquire synchronized images of a calibration board of known size captured by multiple cameras, establish image associations according to synchronized acquisition batches, and generate synchronized acquisition data. The synchronized acquisition batches correspond to a static state of the calibration board. The synchronized acquisition data retains the camera identifiers, calibration board geometric information, and calibration board motion mechanism identifiers associated with the images. The calibration board geometric information includes the physical coordinates of calibration feature points and the distance between points. Based on the synchronously acquired data, the independent verification records of the calibration plate motion mechanism are matched, the relative motion error boundary is extracted and associated with the synchronously acquired data to generate motion constraint acquisition data. The independent verification records are formed using measurement results independent of the multiple cameras to be calibrated, and their applicable scope covers the calibration working range of the calibration plate motion mechanism. The relative motion error boundary is determined based on the repetitive motion deviation, measurement uncertainty and preset error margin, limiting the allowable deviation of relative translation and relative rotation in the calibration plate coordinate system. A reference camera is selected based on the motion constraint acquisition data, a unified calibration reference is established and associated with the motion constraint acquisition data, and calibration base data is generated. The unified calibration reference uses the reference camera coordinate system as the unified coordinate system, fixes the pose of the reference camera, and determines the spatial scale using the distance between points in the calibration plate geometric information.
[0024] In this embodiment, obtaining the calibration observation set containing positioning uncertainty specifically includes: The calibration base data is input into the machine learning observation model, and feature encoding is performed on the synchronous image while retaining the acquisition association to generate an observation feature set. The machine learning observation model includes a shared image coding network, a position prediction branch, and an uncertainty prediction branch. The shared image coding network is composed of an input mapping layer, a downsampling coding layer, and an upsampling recovery layer connected in sequence. The input mapping layer converts the synchronous image into an initial feature map through convolution. The downsampling coding layer extracts the local contour and neighborhood arrangement features of the calibration feature points step by step, reduces the spatial resolution of the feature map, and retains the output of each coding layer. The upsampling recovery layer increases the spatial resolution of the feature map step by step. The recovered feature map is channel-stitched and convolved with the feature map of the coding layer with the same resolution to form observation features. The mapping relationship between the position of the observation features and the position of the original synchronous image is established according to the sampling ratio and position offset of each layer. The observation feature set retains the association between the observation features and the synchronous acquisition batch and camera identifier. The observed feature set is fed into the location prediction branch and the uncertainty prediction branch to determine the location of the calibrated feature points and estimate the location uncertainty, generating location observation data. The location prediction branch predicts the location response and local coordinate offset through convolution mapping, extracts the peak value of the location response and superimposes the local coordinate offset, and converts it into the sub-pixel position in the original synchronous image coordinate system according to the mapping relationship. The uncertainty prediction branch reads the local observation features corresponding to the location of the calibrated feature points, outputs the parameters of the two-dimensional lower triangular matrix through parametric regression, performs an exponential transformation on the diagonal elements, and multiplies the two-dimensional lower triangular matrix with the transpose of the two-dimensional lower triangular matrix to obtain the two-dimensional location error covariance describing the location uncertainty. The diagonal elements represent the lateral location error variance and the longitudinal location error variance, respectively, and the off-diagonal elements represent the covariance between the lateral location error and the longitudinal location error. The machine learning observation model is pre-trained using training images with the true values of the calibrated feature point locations, calculates the deviation between the predicted position and the true position, and uses the weighted square of the position deviation under the inverse matrix of the two-dimensional location error covariance and the logarithm of the determinant of the two-dimensional location error covariance to constitute the training loss. Based on the positioning observation data and the geometric information of the calibration board, calibration feature points are matched and calibrated. According to the collection and matching results, a calibration observation set containing positioning uncertainty is generated. The matching is based on the arrangement relationship of feature points recorded by the geometric information of the calibration board, and the position of the calibration feature points in the image is associated with the physical coordinates under the coordinate system of the calibration board. The matching result retains the positioning uncertainty, synchronous acquisition batch and camera identifier. The calibration observation set inherits the unified calibration benchmark and relative motion error boundary in the calibration basic data.
[0025] In this embodiment, obtaining the initial calibration state specifically includes: Read the calibration observation set, configure camera parameters and inner layer pose according to the unified calibration reference, establish parameter sharing relationship across acquisition batches and perform initialization to form calibration parameter configuration. The camera parameters include camera intrinsic parameters, distortion parameters and rotation and translation parameters relative to the unified calibration reference. The camera parameters of the same camera are shared between synchronous acquisition batches. The inner layer pose is the position and orientation of the calibration board relative to the unified calibration reference. Cameras in the same synchronous acquisition batch share an inner layer pose. The camera parameters and inner layer pose are initialized according to the projection correspondence between the calibration feature point position and physical coordinates. The pose of the reference camera remains fixed. The projection positions of calibration feature points are calculated based on the calibration parameters. The reprojection deviation is constructed using the projection positions and the positions of calibration feature points in the calibration observation set. The reprojection deviation is weighted and summarized according to the positioning uncertainty to establish the Wiberg calibration target. The projection positions are transformed to a unified calibration reference by the physical coordinates of the calibration board through the inner layer pose. The coordinate transformation and imaging relationship determined by the camera parameters are then calculated. For each reprojection deviation, the weighted sum is calculated using the inverse matrix of the two-dimensional positioning error covariance. The weighted sums of the cameras in the synchronous acquisition batch are collected and summed across batches. The positioning uncertainty remains unchanged during one inner layer pose solution. For Wiberg calibration targets with fixed camera parameters, an inner-layer pose solving problem is constructed according to synchronous acquisition batches. Common observation constraints for sharing the inner-layer pose are collected, and the inner-layer pose is iteratively updated to generate an inner-layer solution state. The inner-layer pose solving problem uses the camera in the same synchronous acquisition batch to observe the common constraint calibration board pose. The pose increment equation is constructed by calculating the derivative of the reprojection deviation with respect to the inner-layer pose. Damping is applied to the pose increment equation and the local rotation increment and translation increment are solved. The candidate update is substituted back into the Wiberg calibration target. When the target value decreases, the candidate update is accepted. After normalization according to the preset parameter scale, the inner-layer iteration ends when the gradient norm and the pose increment norm meet the preset threshold requirements. The inner-layer solution state retains the obtained inner-layer pose and the corresponding camera parameters. An inner-layer stationary point equation is established based on the inner-layer solution state. The inner-layer pose and camera parameters are differentiated separately to construct the inner-layer pose response equation and record the response solution state. The initial calibration state is generated by associating the inner-layer solution state. The inner-layer stationary point equation is constructed with the first derivative of the Wiberg calibration target with respect to the inner-layer pose set to zero. The inner-layer pose response equation is formed by the two types of derivatives of the inner-layer stationary point equation. The solution result describes the pose change required to maintain the inner-layer stationary point condition when the camera parameters undergo local changes. The pose response is calculated when the derivative matrix of the stationary point equation with respect to the inner-layer pose satisfies the full rank requirement and the condition number does not exceed the preset upper limit. When both the full rank requirement and the upper limit requirement of the condition number are not satisfied, a response degradation flag is recorded. The initial calibration state saves the Wiberg calibration target, the inner-layer solution state, the inner-layer stationary equation, and the response solution state.
[0026] This invention incorporates positioning uncertainty and cross-camera shared inner-layer pose into the Wiberg calibration objective, which can distinguish the reliability of observation constraints, limit the inconsistency of pose estimation under the same calibration board state, and reflect the correlation between camera parameters and calibration board pose through inner-layer pose response, reduce the interference of local positioning deviation on parameter estimation, and provide a consistent geometric solution basis for competitive branch identification.
[0027] In this embodiment, obtaining the competitive pose set specifically includes: The initial calibration state is read, the inner stationary point equation and the obtained pose solution are extracted, a known root record is established according to the synchronous acquisition batch, the inner stationary point equation after the known root is excluded is constructed, and the exclusion solution state is formed. The known root record associates the obtained pose solution with fixed camera parameters. The known root exclusion is performed on the same synchronous acquisition batch. First, the relative rotation angle and translation distance between the pose to be solved and the known pose solution are scaled and normalized respectively. Then, the exclusion factor is constructed according to the normalized distance. The exclusion factor is formed by taking the reciprocal of the normalized distance to a preset power of not less than one and superimposing a preset positive bias. The exclusion factor corresponding to the known root record is multiplied together and applied to the inner stationary point equation. The position where the normalized distance is zero is not used as the solution starting point. The inner stationary point search is performed based on the exclusion solution status. The search results are substituted back into the inner stationary point equation without the exclusion factor for stationary point verification. The new pose solution that passes the stationary point verification is added to the known root record and the exclusion solution status is updated. The obtained pose solutions and the new pose solutions are collected to generate a candidate stationary point set. The inner stationary point search keeps the camera parameters unchanged and applies a non-zero local perturbation to the known pose solution to form the search starting point. The search results are obtained by iteratively solving the excluded inner stationary point equation. The stationary point verification requires that the residual of the inner stationary equation without the exclusion factor meets the preset stationary point threshold. When the normalized distance between the new pose solution and the known root record exceeds the preset deduplication threshold, the known root record is added. The search ends after the preset number of searches is reached. For the candidate stationary point set, the observation error bound is determined according to the positioning uncertainty. The candidate stationary points are substituted back into the Wiberg calibration target, the reprojection error is calculated, and the local minima check and the joint observation physical feasibility check are performed to generate the pose verification state. The observation error bound uses the covariance weighted projection error as the comparison scale. The preset quantile value of the single-item weighted square quantity is determined according to the positioning deviation of the independent verification sample based on the machine learning observation model. The batch error upper limit is formed by summing the effective observation items of the synchronous acquisition batch. The local minima check checks the inner layer pose gradient and curvature under fixed camera parameters, excludes stationary points with negative curvature direction, and performs local perturbation verification for zero curvature direction. The joint observation physical feasibility check requires that the same synchronous acquisition batch shares an inner layer pose and checks that the calibration feature points participating in the observation have positive depth in the corresponding camera coordinate system and that the projection is located in the effective imaging area. The pose verification status is read, and poses that pass the local minima verification and the common observation physical feasibility verification and meet the observation error bound are retained. Pose deduplication is performed, and the inner pose response equation is reconstructed to retain the poses and the response solution status is recorded to form a competitive pose set. The observation error bound is compared with the weighted projection error of the same synchronous acquisition batch in the Wiberg calibration target. Stationary points where local disturbances cause a decrease in target value are not retained. Pose deduplication uses the normalized distance of the known root record and the preset deduplication threshold. The inner pose response equation is re-derived from the inner stationary point equation without the exclusion factor. The competitive pose set retains the correlation between pose, camera parameters, synchronous acquisition batch and response solution status. The number of poses that meet the retention conditions is a single pose record retained at a time.
[0028] This invention applies known root exclusion to the inner stationary point equations and retains the calibration target without exclusion factors as the basis for pose verification. This reduces the situation where the search repeatedly returns to the known pose, and avoids using the numerical changes caused by exclusion factors as the basis for calibration accuracy. This allows competing poses with similar reprojection errors to be retained and compared, reducing the risk that pose ambiguity will be masked by a single solution result.
[0029] In this embodiment, obtaining the competitive calibration branch set specifically includes: Read the competitive pose set, organize calibration branches according to the synchronous acquisition batch and camera parameters, determine the camera parameter trial direction and configure the path advance, establish the joint extension relationship between the inner pose and the path advance, and generate the extension initial state. The calibration branch selects a reserved pose in each synchronous acquisition batch. The same calibration branch uses a set of shared camera parameters. The path advance controls the camera parameters to change from the current parameters along the trial direction. When the path advance is zero, it corresponds to the current camera parameters. When it is one, it corresponds to the complete trial update. The trial direction is determined according to the Wiberg calibration target and the response solution state. When there is a response degradation indicator, the trial path is constructed by the camera parameter descent direction under the fixed inner pose condition. The joint extension relationship adopts the inner stationary point equation without the exclusion factor and keeps the reference camera pose fixed. For the initial state of the extension, the tangential direction of the joint extension relationship is calculated. The inner layer pose and path advance are predicted along the tangential direction. Joint correction is performed by combining the inner layer stationary point equation and pseudo-arc length constraint. The correction result and the branch support connection relationship are recorded to generate the pose extension state. The tangential direction makes the first-order change of the inner layer stationary point equation zero and normalizes it according to the preset parameter scale. The initial extension selects the tangential direction with the increased path advance. The continuous extension keeps the tangential direction consistent with the previous tangential direction. The pseudo-arc length constraint requires that the projection of the displacement between the correction result and the prediction result on the predicted tangential direction be zero. The joint correction adjusts the inner layer pose and path advance at the same time. The path advance is allowed to change the direction of increase or decrease along the solution branch turning point. When the correction residual does not meet the preset correction threshold, the extension step size is reduced and the prediction is re-predicted. When the correction cannot be completed even after reaching the preset minimum step size, the last branch that passed the correction and the incomplete mark are retained. Candidate camera parameters are recovered based on the pose extension state. The inner pose is verified according to the branch support connection relationship. The outer update is tested under the Wiberg calibration target. The verified camera parameters are associated with the inner pose to generate a calibration branch set. The branch support connection relationship records the pose change, path advance change and tangential orientation between adjacent extension points. The inner pose verification requires that the correction result is within the preset prediction neighborhood and is verified by the inner stationary point equation without the application of exclusion factors. The outer update test requires that the inner pose under the candidate camera parameters passes the local minima test and the joint observation physical feasibility test, and that the Wiberg calibration target value meets the preset descent condition. Stationary points that are only used for extension tracking but fail the outer update test are not considered as accepted calibration results. Branches that have not completed extension retain their original valid records. The calibration branch set is read, and the reprojection error of each calibration branch is calculated using the same calibration observation set. Valid branches are retained according to the observation error bound. The differences in camera parameters of the valid branches are compared according to the calibration tolerance. Valid branches with out-of-tolerance terms are grouped into competing branch pairs to generate a competing calibration branch set. The observation error bound is verified according to the synchronous acquisition batch. During the branch comparison, the position and positioning uncertainty of the calibration feature points remain unchanged. The calibration tolerance limits the camera intrinsic parameter deviation, distortion parameter deviation, relative rotation deviation, and relative translation deviation. When any comparison term exceeds the corresponding calibration tolerance, a competing branch pair is formed. The competing calibration branch set retains the association between the competing branch pair and the camera parameters, inner layer pose, and branch support connection relationship. When the competing branch pair is empty, the calibration branch record that has passed the observation error bound verification is retained.
[0030] This invention incorporates the inner layer pose and path advancement into the joint extension relationship, and constrains the outer layer update with the branch connection relationship. It can maintain continuous tracking of the solution near the branch turning point that meets the extension condition, reduce erroneous updates caused by pose jumps, and combine the observation error boundary and calibration tolerance to distinguish the fitting effectiveness and parameter divergence, thereby improving the reliability of competitive calibration branch comparison.
[0031] In this embodiment, obtaining the calibration iteration state specifically includes: Read the competitive calibration branch set, generate candidate paired motions within the common visible range and the reachable range of the motion mechanism, fix the camera parameters of the competitive branch pairs, and call the Wiberg inner layer solution to perform bidirectional refitting under the constraint of relative motion error bound, generating bidirectional refitting results. The candidate paired motions are determined by the initial acquisition pose and relative motion command. The bidirectional refitting uses the two calibration branches of the competitive branch pairs as the prediction branch and the fitting branch, respectively. The prediction branch generates the paired projection positions of the calibration feature points according to the candidate paired motions. The fitting branch adjusts the initial inner layer pose and relative motion deviation to reduce the weighted difference of the paired projection positions. The relative motion deviation is limited within the relative motion error bound. The two fittings use the same camera observation terms and positioning uncertainty weighting scale. The bidirectional refitting results record the fitting convergence state and the remaining observation differences after refitting. Based on the bidirectional refit results, the minimum bidirectional residual observation difference between the competing branch pairs of each candidate pair of motions is taken. The candidate pair of motion with the largest minimum value is selected, and the calibration plate motion mechanism is controlled to execute the pair of motions and trigger synchronous acquisition to generate a set of supplementary images. The minimum value is selected from the competing branch pairs and two fitting directions associated with the same candidate pair of motions. Candidate pair of motions that have not completed the bidirectional fitting convergence verification are not selected. The calibration plate motion mechanism stops moving at the initial acquisition pose and the acquisition pose after the relative motion command is executed. After the calibration plate stabilizes, multi-camera synchronous acquisition is triggered. The supplementary image set records the correlation between the pair acquisition batch, camera identification, relative motion command, and relative motion error boundary. The re-captured image set is input into the machine learning observation model to update the calibration observation set. Paired inner layer poses are associated according to the relative motion error boundary. The shared initial pose and constrained relative motion are reconstructed into inner layer variables of the paired acquisition group to reconstruct the Wiberg calibration target and form an associated calibration state. The calibration observation set is updated by performing calibration feature localization, localization uncertainty estimation and calibration board physical coordinate matching on the re-captured images. The shared initial pose is the initial calibration board pose used by the paired acquisition group. The inner layer pose of the second acquisition is determined by the shared initial pose and the constrained relative motion. The constrained relative motion is determined by the relative motion command and the motion deviation within the relative motion error boundary. The reconstructed inner layer solution uses the paired images to constrain the shared initial pose and motion deviation. The paired acquisition group no longer has two sets of inner layer poses that can change independently. The associated calibration state is read, the inner layer pose solution is re-executed and the initial calibration state is updated. The known root elimination, pseudo-arc length extension and competitive branch determination are repeatedly executed. For the retained competitive branch pairs, bidirectional refitting and paired motion re-shooting are performed iteratively to form a calibration iteration state. The initial calibration state is updated with the inner layer stationary point equation and response solution state according to the reconstructed Wiberg calibration target. The original pose solution is used as the initial value for re-solving. After verification by the updated inner layer stationary point equation, the known root record is added. The competitive branch determination uses the calibration observation set formed by the actual re-shot images. Branches whose reprojection error after fitting exceeds the observation error limit are not retained. When the competitive branch pair is empty, the divergence elimination state is recorded. When the preset re-shooting limit is reached and there are still competitive branch pairs, the divergence not eliminated state is recorded. The calibration iteration state retains the updated calibration observation set, camera parameters, inner layer pose and divergence determination results.
[0032] This invention combines bidirectional refitting under motion error constraints with paired inner-layer variable recombination. The reshooting focuses on the smallest branch difference that is still retained after pose compensation, making new observations more targeted. At the same time, by sharing the initial pose and restricting independent pose adjustment by constrained relative motion, the absorption of camera parameter deviations is enhanced, the ability to distinguish competitive calibration branches is improved, and the stability of automatic calibration results is increased.
[0033] In this embodiment, obtaining the multi-camera calibration parameters under the unified calibration benchmark specifically includes: Read the calibration iteration status, check the divergence judgment results, and review the reprojection error of the retained calibration branches according to the observation error boundary. Extract the calibration parameters to be verified from the calibration branches that have passed the review and whose divergence has been eliminated. The divergence elimination requires that there are no competing branch pairs between the retained calibration branches whose camera parameter differences exceed the calibration tolerance. The reprojection error is calculated using the updated calibration observation set according to the synchronous acquisition batch. The calibration branches that meet the requirements are arranged in ascending order according to the Wiberg calibration target value. Extract the camera parameters of the first calibration branch. If no valid calibration branch is retained, record the verification failure status. The calibration parameters to be verified are fixed, and synchronous images of the known-size calibration board that were not involved in the calibration solution are read. Cross-camera calibration feature points are matched and spatial coordinates are reconstructed. The distance between the reconstructed points is compared with the distance between the points in the calibration board's geometric information to generate a calibration completion verification result. The synchronous images that were not involved in the calibration solution are collected at preset verification locations. The positions of calibration feature points are extracted through a machine learning observation model. During the verification, the calibration parameters to be verified remain unchanged. The spatial coordinates are determined by the cross-camera projection relationship of the corresponding calibration feature points. The absolute deviation of the distance between points is calculated one by one according to the point pairs recorded in the calibration board's geometric information. When all point pairs at the preset verification locations are reconstructed and the absolute deviation of the distance does not exceed the preset length verification threshold, the verification pass status is recorded. The preset length verification threshold is determined according to the spatial measurement accuracy required by the calibration task. Read the calibration completion verification result. If the verification is successful, organize the calibration parameters to be verified according to the camera identifier, associate them with the unified calibration reference, and output the multi-camera calibration parameters. The multi-camera calibration parameters include the camera intrinsic parameters, distortion parameters, and rotation and translation parameters relative to the unified calibration reference of each camera. When outputting, keep the pose of the reference camera fixed and use the spatial scale determined by the calibration basis data. If the verification fails, retain the calibration iteration status and the verification failure record, and do not output the calibration completion result.
[0034] Example 1: To verify the feasibility of the present invention in practice, the present invention was applied to the multi-camera automatic calibration scenario of a three-dimensional dimension inspection device for an industrial workpiece. By unifying the imaging parameters and spatial position relationships of different cameras, a geometric basis was provided for workpiece dimension measurement, hole positioning, and assembly deviation detection.
[0035] The testing equipment is equipped with four fixed industrial cameras, all forming a common field of view within the testing area. A calibration plate of known dimensions is mounted on a movable mechanism whose position and orientation can be adjusted. The physical coordinates of the calibration feature points and the distances between them are determined through independent measurements. Each camera synchronously acquires images after the calibration plate stops moving.
[0036] Under different viewing angles, the accuracy of calibration feature points varies due to perspective changes, lighting differences, and local occlusion. When the calibration board is imaged close to the front, different poses may produce similar reprojection errors, making it difficult for local optimization results to fully reflect the accuracy of spatial calibration. Repeatedly adding images from similar positions may not eliminate pose ambiguity.
[0037] Before calibration, an independent measuring device is used to verify the motion mechanism, and the relative motion error boundary is determined by combining the repetitive motion deviation, measurement uncertainty, and error margin. Synchronous images are used to select a reference camera and fix its pose according to the camera identification associated with the batch and the geometric information of the calibration board. The spatial scale is determined by the actual distance between points on the calibration board, forming the basic calibration data.
[0038] A machine learning observation model extracts the locations of calibration feature points and predicts the 2D positioning error covariance. The positioning results are then correlated with the physical coordinates of the calibration board to form a calibration observation set. Model parameters remain fixed during the comparison period, and positioning uncertainty is incorporated into the observation weighting of the Wiberg calibration target, thus controlling the impact of disturbed observations on parameter estimation. The same synchronous acquisition batch shares the inner-layer pose of the calibration board, and camera parameters from the same camera are shared between acquisition batches.
[0039] After obtaining the initial results from the inner layer pose solution, the known root exclusion is performed on the inner layer stationary point equations to search for candidate solutions different from the known poses. The stationary point equations without exclusion factors are then used to verify the results against the calibration target. The candidate poses are simultaneously checked for positive depth and effective imaging area. The results that pass the verification are retained as competing poses to avoid determining calibration parameters based solely on a single local convergence result.
[0040] When competing poses are used to update camera parameters, pseudo-arc length extensions are performed on the inner-layer pose and path advance, and the continuity of the update is checked through the branch connection relationship. After the outer-layer solution is completed, a competitive calibration branch set is formed based on the observation error bound and calibration tolerance. Branches that fail to extend or whose solutions are incomplete retain status indicators and are not directly considered as excluded.
[0041] When competing branch pairs exist, candidate paired motions are set within the common visible range and the reachable range of the motion mechanism. Two-way refitting allows the fitted branches to adjust the initial pose and motion deviation within the error bound. The value of reshooting is evaluated by the observation difference that is still retained after adjustment, and the candidate motion with the smallest remaining observation difference is selected for reshooting.
[0042] After the re-captured images update the calibration observation set, the paired inner layer poses are jointly determined by the shared initial pose and constrained relative motion, limiting the compensation of camera parameter deviations by independent pose adjustments. The recombined inner layer variables participate in the solution again, updating the stationary point equations, pose response, and known root records until the calibration completion verification conditions are met.
[0043] Thirty test conditions were set up for comparative verification, covering clear imaging, illumination variations, and small tilt angles with local occlusion. All schemes used the same initial images, initialization methods, acquisition budget limits, and computing equipment. The effectiveness of conventional Wiberg calibration, machine learning weighting, known root exclusion, pseudo-arc length extension, and the complete scheme were compared.
[0044] Independent verification uses calibration board images not involved in the solution process. Feature point spatial coordinates are reconstructed with fixed camera parameters and then compared with independently measured point distances. No known size constraints are used to correct the results during reconstruction. Error statistics include tests that failed verification. Online computation time covers feature localization, calibration solution, and re-enhancing evaluation. Total calibration time also includes acquisition and motion waiting time. Consistent statistical methods are used for all schemes. Specific comparison data are shown in Table 1. Table 1 Comparison of geometric accuracy and computational cost of different multi-camera automatic calibration strategies
[0045] As shown in Table 1, compared with the machine learning weighted Wiberg calibration scheme, the complete scheme reduces the root mean square error of length for independent verification by approximately 79.1%, and increases the verification pass rate from 66.7% to 96.7%. Compared with the scheme that incorporates known root exclusion and pseudo-arc length extension, the root mean square error of length is further reduced by approximately 51.7%. At the same time, the average number of images solved is reduced, reflecting the effect of targeted reshooting and paired motion constraints. The increased computation time reflects the additional computational overhead brought about by competitive branch search and reshoot evaluation.
[0046] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A high-precision automatic calibration method for multiple cameras based on machine learning, characterized in that, Includes the following steps: Acquire synchronized images of a calibration board of known size from multiple cameras and the relative motion error boundary, establish a unified calibration benchmark, and generate calibration basis data; The calibration baseline data is input into the machine learning observation model to generate a calibration observation set containing positioning uncertainty; Establish a Wiberg calibration target using the calibration observation set, perform inner layer pose solving, and generate the initial calibration state; Read the initial calibration state, perform known root elimination on the inner stationary point equation, determine the observation error boundary according to the positioning uncertainty, verify the pose according to the Wiberg calibration target, and form a competitive pose set; For the competitive pose set, pseudo-arc length extension is performed on the inner pose and path advance, the outer update is checked according to the sub-support connection relationship, competitive branches are screened according to the observation error boundary and calibration tolerance, and a competitive calibration branch set is generated. Perform bidirectional refitting under relative motion error bound constraints on the competitive calibration branch set, select the pair of motion reshoots with the smallest remaining observation difference, update the calibration observation set, associate the pair of inner layer poses according to the relative motion error bound and reorganize the inner layer variables, update the initial calibration state and iterate to form the calibration iteration state; Check the calibration iteration status, and output the multi-camera calibration parameters under the unified calibration benchmark after verification of calibration completion.
2. The high-precision automatic calibration method for multiple cameras based on machine learning according to claim 1, characterized in that, The specific methods for obtaining the calibration baseline data include: Acquire synchronized images of a calibration board of known size from multiple cameras, establish image associations according to the synchronized acquisition batches, and generate synchronized acquisition data; Based on the independent verification records of the calibration plate motion mechanism matched with the synchronously acquired data, the relative motion error boundary is extracted and associated with the synchronously acquired data to generate motion constraint acquisition data; A reference camera is selected based on the motion constraint acquisition data, a unified calibration benchmark is established and associated with the motion constraint acquisition data, and calibration basis data is generated.
3. The high-precision automatic calibration method for multiple cameras based on machine learning according to claim 1, characterized in that, The specific steps involved in obtaining the calibration observation set containing positioning uncertainty are as follows: The calibration base data is input into the machine learning observation model, and feature encoding is performed on the synchronized images while retaining the acquisition correlation to generate an observation feature set. The observed feature set is fed into the location prediction branch and the uncertainty prediction branch to determine the location of the calibration feature points and estimate the positioning uncertainty, thereby generating positioning observation data. Based on the positioning observation data and the geometric information of the calibration board, calibration feature points are matched and then generated according to the collected and associated matching results, resulting in a calibration observation set containing positioning uncertainty.
4. The high-precision automatic calibration method for multiple cameras based on machine learning according to claim 1, characterized in that, Obtaining the initial calibration state specifically includes: Read the calibration observation set, configure the camera parameters and inner layer pose according to the unified calibration benchmark, establish parameter sharing relationship across acquisition batches and perform initialization to form the calibration parameter configuration; The projection positions of calibration feature points are calculated based on the calibration parameters. The reprojection deviation is constructed using the projection positions and the positions of calibration feature points in the calibration observation set. The reprojection deviation is weighted and summarized according to the positioning uncertainty to establish the Wiberg calibration target. For Wiberg calibration target with fixed camera parameters, an inner layer pose problem is constructed according to synchronous acquisition batches. Common observation constraints for sharing the inner layer pose are collected, the inner layer pose is iteratively updated, and the inner layer solution state is generated. Based on the inner layer solution state, establish the inner layer stationary point equation, differentiate with respect to the inner layer pose and camera parameters respectively, construct the inner layer pose response equation and record the response solution state, and generate the initial calibration state by associating the inner layer solution state.
5. The high-precision automatic calibration method for multiple cameras based on machine learning according to claim 1, characterized in that, The acquisition of the competitive pose set specifically includes: Read the initial calibration state, extract the inner layer stationary point equations and the obtained pose solutions, establish known root records according to the synchronous acquisition batches, construct the inner layer stationary point equations after excluding known roots, and form the excluded solution state. Based on the exclusion solution state, perform inner stationary point search, substitute the search results back into the inner stationary point equation without the exclusion factor for stationary point verification, add the new pose solution that passes the stationary point verification to the known root record and update the exclusion solution state, and gather the obtained pose solutions and the new pose solutions to generate a candidate stationary point set. For the candidate stationary point set, the observation error boundary is determined according to the positioning uncertainty. The candidate stationary points are substituted back into the Wiberg calibration target, the reprojection error is calculated, and the local minima verification and the physical feasibility verification of the common observation are performed to generate the pose verification state. Read the pose verification status, retain the poses that pass the local minima verification and the common observation physical feasibility verification and satisfy the observation error bound, perform pose deduplication, reconstruct the inner pose response equation for the retained poses and record the response solution status to form a competitive pose set.
6. The high-precision automatic calibration method for multiple cameras based on machine learning according to claim 1, characterized in that, The specific steps involved in obtaining the competitive calibration branch set are as follows: Read the competitive pose set, organize the calibration branch according to the synchronous acquisition batch and camera parameters, determine the camera parameter trial direction and configure the path advance, establish the joint extension relationship between the inner pose and the path advance, and generate the extension initial state; For the initial state of the extension, calculate the tangential direction of the joint extension relationship, predict the inner layer pose and path advance along the tangential direction, perform joint correction by combining the inner layer stationary point equation and pseudo-arc length constraint, record the correction results and the sub-support connection relationship, and generate the pose extension state. Candidate camera parameters are recovered based on pose extension state, inner layer pose is verified according to branch connection relationship, outer layer update is checked under Wiberg calibration target, and the verified camera parameters are associated with inner layer pose to generate calibration branch set; Read the calibration branch set, calculate the reprojection error of each calibration branch using the same calibration observation set, retain the effective branches according to the observation error boundary, compare the differences in camera parameters of the effective branches according to the calibration tolerance, form competitive branch pairs with out-of-tolerance terms, and generate a competitive calibration branch set.
7. The high-precision automatic calibration method for multiple cameras based on machine learning according to claim 1, characterized in that, The specific steps for obtaining the calibration iteration state include: Read the competitive calibration branch set, generate candidate pairs of motion within the common visible range and the reachable range of the motion mechanism, fix the camera parameters of the competitive branch pairs, and call the Wiberg inner layer solver under the relative motion error bound to perform bidirectional refitting and generate bidirectional refitting results. Based on the bidirectional refit results, the minimum bidirectional residual observation difference between the competing branch pairs of each candidate pair motion is taken, and the candidate pair motion with the largest minimum value is selected. The calibration plate motion mechanism is controlled to execute the pair motion and trigger synchronous acquisition to generate a set of supplementary images. The re-captured image set is input into the machine learning observation model to update the calibration observation set. The inner layer poses are associated according to the relative motion error boundary. The inner layer variables of the pair acquisition group are reconstructed by sharing the initial pose and the constrained relative motion, and the Wiberg calibration target is reconstructed to form the associated calibration state. Read the associated calibration state, re-execute the inner layer pose solution and update the initial calibration state, repeat the known root elimination, pseudo-arc length extension and competitive branch determination, and perform bidirectional refitting and paired motion supplementation on the retained competitive branches to form the calibration iteration state.
8. The high-precision automatic calibration method for multiple cameras based on machine learning according to claim 1, characterized in that, The specific methods for obtaining the multi-camera calibration parameters under the unified calibration benchmark include: Read the calibration iteration status, check the divergence judgment results, verify the reprojection error of the retained calibration branch according to the observation error boundary, and extract the calibration parameters to be verified from the calibration branch that has passed the verification and the divergence has been eliminated. Fix the calibration parameters to be verified, read the synchronous image of the known size calibration board that has not participated in the calibration solution, match the cross-camera calibration feature points and reconstruct the spatial coordinates, compare the distance between the reconstructed points with the distance between the points in the geometric information of the calibration board, and generate the calibration completion verification result. Read the calibration completion verification results. When the verification is successful, organize the calibration parameters to be verified according to the camera identification, associate them with the unified calibration benchmark, and output the calibration parameters of multiple cameras.
