Self-calibration method and system for multi-camera measurement system

Abstract

The present invention discloses a self-calibration method and system for a multi-camera measurement system, which belongs to the technical field of camera calibration. The method comprises the following steps: arranging a calibration plate with coded marking points and non-coded marking points within a measurement field of view, collecting images of the calibration plate in different postures, establishing an error equation and performing global optimization to obtain the intrinsic parameter value of each camera; taking the center point of the calibration plate as the world coordinate origin, and obtaining initial extrinsic parameters by reconstructing the marking points of the cameras within a common field of view; arranging coded marking points after removing the calibration plate, reconstructing the coordinates of the marking points using existing extrinsic parameter cameras, and solving the initial values ​​of the extrinsic parameters of other cameras; establishing error equations for all cameras, and obtaining accurate extrinsic parameters with the constraint that the coordinates of the center of gravity of the marking points remain unchanged; the method uses coded marking points and a marking point ruler to overcome the problems of difficulty in solving the extrinsic parameters of other cameras without a common field of view and high operating cost, while providing accurate initial values ​​of the extrinsic parameters of all cameras for subsequent calibration.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a self-calibration method and system for a multi-camera measurement system. Background Art

[0002] Camera calibration is a key step in visual measurement, used to determine the camera's intrinsic and extrinsic parameters. These include focal length, distortion, and principal point deviation, while extrinsic parameters include the rotation and translation matrices between the camera coordinate system and the world coordinate system.

[0003] The commonly used camera intrinsic and extrinsic parameter calibration methods in the industry include: first, placing a calibration plate with known real-world coordinates of feature points on the calibration plate in the common field of view of all cameras; then, all cameras synchronously collect and reconstruct feature points; finally, establishing a matrix equation between the real-world coordinates of the feature points and the corresponding image coordinates captured by the camera to solve the intrinsic and extrinsic parameters of each camera.

[0004] This calibration method has the following disadvantages: Technical limitations: The existing technical limitations are mainly reflected in the following two aspects. First, it is highly dependent on the format of the calibration plate and the public field of view of the camera. The existing calibration technology requires the use of a camera with a public field of view to capture a calibration plate that matches the measurement format in order to solve the camera's internal and external parameters. The strict operating requirements make the actual measurement scenarios in which the existing calibration method can be used very limited. Second, repeated calibration leads to coordinate system conversion problems: Using existing technology to calibrate multiple times in succession can easily lead to inconsistent coordinate system origins. This is especially time-consuming and difficult for complex automated measurement systems, as each measurement requires unified positioning of multiple coordinate systems. Environmental impact: The calibration of a multi-camera measurement system is inevitably affected by external factors. Specifically, 1) Vibration impact: The measurement environment of a multi-camera measurement system inevitably has environmental vibrations. The environmental vibrations are transmitted through the structural components that fix the cameras, causing changes in the relationship between the camera's external parameters. Existing seismic mitigation solutions (such as using highly seismic-resistant materials) can mitigate the impact of environmental factors to a certain extent, but the hardware cost is too high and their applicability is limited. 2) Material thermal expansion and contraction: During long-term measurements, heat generated by the camera is transferred to the metal structural components that secure the camera, causing thermal expansion and deformation of the structural components, thereby changing the camera's extrinsic parameters. Selecting materials with lower linear expansion coefficients for structural components can reduce the extent of thermal expansion and deformation, but it cannot completely eliminate it and increases the system's hardware cost. Intrinsic accuracy degradation: Over extended use of a multi-camera measurement system, the rigidity and inherent stress of the structural components will affect the camera's extrinsic parameters, thereby reducing the system's measurement accuracy. Specifically, insufficient material rigidity: The materials used to stabilize the structure of a multi-camera measurement system are mostly rigid (typically steel frames, aluminum alloys, iron frames, etc.). The larger the field of view, the higher the rigidity requirement. Currently, it is difficult to find materials that meet the rigidity requirements for large field of view measurements. Second, self-stress release: After a multi-camera measurement system has been running for a long time, the rigid components in the system will undergo varying degrees of deformation due to the continuous effects of their own gravity and internal stress, causing changes in the external parameter relationships of the camera. This stress deformation comes from the material itself and is difficult to completely eliminate. Summary of the Invention

[0005] On the one hand, in order to solve the problems of the prior art, the present invention provides a self-calibration method for a multi-camera measurement system, the method comprising the following steps:

[0006] Step 1: Intrinsic parameter optimization: Place a calibration plate with coded and non-coded markers within the measurement field of view. Use multiple cameras to synchronously capture images of the calibration plate in different poses. Establish an error equation based on the resection method and perform global optimization to obtain the intrinsic parameter values ​​of each camera.

[0007] Step 2: Solve the initial values ​​of extrinsic parameters: Use the center point of the calibration plate as the world coordinate origin and reconstruct the initial extrinsic parameters by using the marker points of the cameras in the public field of view. After removing the calibration plate, arrange the coded marker points and use the existing extrinsic parameter camera to reconstruct the marker point coordinates. Combined with the cone method, solve the initial values ​​of the extrinsic parameters of other cameras.

[0008] Step 3: Optimize the external parameters: Establish the error equations of all cameras, perform bundle optimization with the constant coordinates of the center of gravity of the marker points as the constraint condition, and obtain accurate external parameters;

[0009] Step 4: Scale correction: perform proportional correction on the optimized external parameters through the scale point spacing ratio;

[0010] Step 5: Dynamic calibration: Before each measurement, control points are collected synchronously, and the relationship between external parameters is dynamically calibrated based on the principle of thermal expansion in combination with ambient temperature data.

[0011] Furthermore, the intrinsic parameter optimization method described in step 1 includes: using a dot matrix calibration plate with marked points, continuously moving the position of the calibration plate within the measurement format, and placing different postures at each position. At the same time, all cameras synchronously capture images of the calibration plate, and then based on the back intersection method, uniformly establish error equations for all cameras, and then bundle and optimize to obtain intrinsic parameter values ​​for all cameras;

[0012] The intrinsic parameter values ​​include: lens focal length , lens distortion ( 、 、 、 、 、 、 ), principal point deviation ( 、 );

[0013] Lens distortion refers to the lens distortion caused by the lens processing technology and the lens assembly position; ( ) represents the mirror distortion parameter; ( ) represents the eccentricity distortion parameter; ( ) represents the plane distortion parameter;

[0014] The principal point deviation indicates the deviation of the camera principal point in the camera coordinate system. Indicates the deviation of the camera principal point position in the width direction in the camera coordinate system; Indicates the deviation of the camera's principal point position in the height direction in the camera coordinate system.

[0015] Furthermore, in step 2, the initial value solution of the extrinsic parameters includes: placing the center point of the dot matrix calibration plate near the center point within the measurement format, and setting the point as the origin of the world coordinate system of the multi-camera measurement system; the point is the point where the center point of the dot matrix calibration plate coincides with the measurement format;

[0016] In a multi-camera measurement system, for cameras with a common field of view, the camera reconstructs the markers on the calibration plate that are located in the common field of view and can be photographed. The resection method is then used to first solve the camera extrinsic parameters involved in the reconstruction of the markers.

[0017] Remove the calibration plate from the measurement area and randomly place non-repeated coded markers. Use a camera with known extrinsic parameters to capture and reconstruct the coordinates of the captured coded markers.

[0018] Using ID matching, determine other cameras that may participate in the reconstruction of the aforementioned marker points, in addition to the camera with existing extrinsic parameters. Based on the cone method and combined with the existing marker coordinates, solve the initial values ​​of the camera extrinsic parameters that may participate in the reconstruction of the marker points.

[0019] The marking points of the dot matrix calibration plate are composed of regularly arranged circular coding points and non-coding points. The spacing between the marking points is fixed and the world coordinates of all the marking points on the calibration plate are known.

[0020] Furthermore, the extrinsic parameter values ​​include: (R, T), representing the matrix transformation relationship from the camera coordinate system to the world coordinate system;

[0021] R represents the rotation matrix relationship from the camera coordinate system to the world coordinate system; T represents the translation matrix relationship from the camera coordinate system to the world coordinate system;

[0022] Based on the intrinsic and extrinsic parameter values, the error equation is: Where X1, X2, and X3 are the intrinsic parameters, extrinsic parameters, and the world coordinate correction values ​​of the measurement point, respectively; A, B, and C are the partial derivative matrices corresponding to the intrinsic parameters, extrinsic parameters, and the world coordinates of the measurement point, respectively; L represents the deviation between the observed real image point coordinates and the theoretical image point coordinate initial values. The corresponding formula is:

[0023] ;

[0024] ;

[0025] Where V represents the camera residual; Represents the component of the camera residual on the X-axis of the image coordinate system; Represents the component of the camera residual on the Y axis of the image coordinate system; Indicates the initial value of the X coordinate of the image point obtained by actual measurement; Indicates the initial value of the X coordinate of the theoretical point image corresponding to the measured image point; Indicates the initial value of the Y coordinate of the image point obtained by actual measurement; Indicates the initial Y coordinate value of the theoretical point image corresponding to the measured image point.

[0026] Furthermore, the resection method is to uniformly establish error equations for all cameras, and bundle and optimize to obtain intrinsic parameter values ​​of all cameras;

[0027] The coordinates of the image points of the marker points on the calibration plate that cover a certain number of images collected by all cameras are used as observation values. The certain number means that all cameras are required to collect calibration plate images at the same time, and the calibration plate images taken by each camera must contain at least 40% of the marker points on the calibration plate. Given the world coordinates of all marker points on the calibration plate, and since each image corresponds to a total of 16 unknown camera intrinsic and extrinsic parameters, the intrinsic and extrinsic parameters of the camera corresponding to the current image can be solved by using the coordinates of the image points of 8 marker points in each image.

[0028] Among them, after the calibration plate moves to different positions, the same camera corresponds to multiple images, and the error equations of multiple images are combined to optimize and solve the optimal initial values ​​of the internal and external parameters of all cameras.

[0029] Furthermore, when the ID matching determines the camera for reconstructing the coded marker points and solves the initial values ​​of the extrinsic parameters of the camera involved in reconstructing the marker points, the coded marker points with non-repeated IDs are randomly arranged within the measurement format;

[0030] The process of solving the initial values ​​of the camera extrinsic parameters that may be involved in the reconstruction of the markers includes: firstly reconstructing the world coordinates of some encoded markers based on the camera with existing extrinsic parameter values;

[0031] Continue to match and determine other cameras that may participate in reconstructing the coded markers in all camera images through the uniqueness of the coded ID, in addition to the cameras with existing parameter values;

[0032] The single image resection method can be used to calculate the initial values ​​of the extrinsic parameters of other cameras that may participate in reconstructing the coded markers. Since the intrinsic parameters of the camera and the world coordinates of the coded markers are known, the corresponding error equation can be expressed as:

[0033] ;

[0034] Where V represents the residual error of the camera; B represents the partial derivative matrix corresponding to the extrinsic parameters; L represents the initial value deviation between the actual measured point image coordinates and the corresponding theoretical point image coordinates; The world coordinate correction number representing the extrinsic parameters;

[0035] Furthermore, the method for solving the initial values ​​of camera extrinsic parameters by single image back intersection is the cone method, which determines the extrinsic parameters corresponding to the camera image by applying the principle that the vertex angles between the image space and the actual object space of the photographic beam cone are equal.

[0036] On the other hand, the present application provides a multi-camera measurement system, the measurement system comprising: a box, a camera, tempered glass, a backlight source, and a controller;

[0037] A plurality of cameras are arranged at intervals around the top inner edge of the box;

[0038] The backlight source is provided on the bottom end surface of the tempered glass, and the coding mark point and the backlight mark point are provided on the top end surface of the tempered glass;

[0039] The tempered glass is placed on the bottom end surface of the box body, and a marking point scale is provided on the top end surface of the tempered glass;

[0040] The controller is connected to the camera and the backlight source respectively.

[0041] Beneficial effects of the present invention:

[0042] The present invention uses coded markers and a marker scale to overcome the difficulties in solving the extrinsic parameters of other cameras without a common field of view and the high operating cost. At the same time, it provides more accurate initial values ​​of the extrinsic parameters of all cameras in the multi-camera measurement system for subsequent calibration. By reconstructing the backlit markers on the measurement plane through the multi-camera measurement system, the problem of system measurement accuracy loss caused by environmental vibration and temperature expansion in the multi-camera measurement system is dynamically calibrated. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the calibration process of the multi-camera measurement system provided by the present invention;

[0044] Figure 2 A schematic diagram of the measurement system structure provided by the present invention;

[0045] Figure 3 This is a schematic image diagram of the photographic beam angle cone provided by the present invention.

[0046] Reference numerals:

[0047] In the figure: 1 is a camera, 2 is a backlight source, 3 is tempered glass, 4 is a coding mark point, 5 is a mark point scale, 6 is a backlight mark point, 7 is a controller, 8 is a center area camera, and 9 is a box. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] See also Figure 1-3 The present invention provides a self-calibration method for a multi-camera measurement system, comprising: placing a calibration plate printed with coded marking points and non-coded marking points at various positions within the measurement field of view, and synchronously capturing images by each camera; establishing error equations for all cameras based on the rear intersection method, and bundling and optimizing the intrinsic parameter values ​​of all cameras.

[0050] The center point of the dot matrix calibration plate is placed near the center point within the measurement format, and this point is set as the world coordinate origin of the multi-camera measurement system; the point is the point of overlap between the center point of the dot matrix calibration plate and the measurement format; at the same time, the cameras with a common field of view in the multi-camera measurement system reconstruct the marker points on the calibration plate, and the rear intersection method is applied to solve the camera extrinsic parameter values ​​involved in the reconstruction of the marker points; the calibration plate is removed and coded marker points with non-repeated numbers are arranged on the measurement format, and the coordinates of the marker points that can be photographed are captured and reconstructed using a camera with existing extrinsic parameters; the number ID matching method is used to determine other cameras that may participate in the reconstruction of the aforementioned marker points in addition to the aforementioned camera, and based on the cone method, the initial values ​​of the camera extrinsic parameters that may participate in the reconstruction of the marker points are solved in combination with the existing marker point coordinates.

[0051] Based on all cameras with initial extrinsic parameter values, a unified error equation is established to bundle and optimize the camera extrinsic parameter values. During the optimization process, the centroid coordinates of all markers remain unchanged as a constraint to ensure that the coordinate system does not shift.

[0052] The above two algorithm processes are repeated until the extrinsic parameter values ​​of all cameras in the multi-camera measurement system are obtained. At the same time, the point distance ratio on the ruler is applied to correct and output the extrinsic parameter values ​​of all cameras in the multi-camera measurement system.

[0053] Using a calibrated multi-camera measurement system, the control points within the measurement format are reconstructed, and the ambient temperature and the mean distance between all control points and the origin of the world coordinate system are recorded to initialize and calibrate the extrinsic parameter relationships of the multi-camera measurement system.

[0054] Before each measurement, all cameras in the multi-camera measurement system synchronously collect and reconstruct the control points within the measurement format based on the initial calibration measurement data. This anchors the world coordinate system, meaning that the origin of the world coordinate system remains stationary. The principle of thermal expansion is then applied to dynamically calibrate the extrinsic parameter relationships of the multi-camera measurement system.

[0055] As a preferred technical solution, the method for calibrating the intrinsic parameters of the multi-camera measurement system includes: first, using a dot matrix calibration plate with marked points, continuously moving the calibration plate position within the measurement format, and placing different postures at each position. At the same time, all cameras of the multi-camera measurement system synchronously capture the calibration plate image; then, based on the back intersection method, a unified error equation is established for all cameras, and the intrinsic parameter values ​​of all cameras are obtained by bundling and optimizing. The intrinsic parameters include: lens focal length ( ), lens distortion ( 、 、 、 、 、 、 ), principal point deviation ( 、 ); External parameters include: ( ), represents the matrix transformation relationship from the camera coordinate system to the world coordinate system; the principal point deviation ( 、 );

[0056] Lens distortion refers to the lens distortion caused by the lens processing technology and the lens assembly position; ( ) represents the mirror distortion parameter; ( ) represents the eccentricity distortion parameter; ( ) represents the plane distortion parameter; T represents the translation matrix relationship from the camera coordinate system to the world coordinate system; R represents the rotation matrix relationship from the camera coordinate system to the world coordinate system; the principal point deviation represents the deviation of the camera principal point in the camera coordinate system, Indicates the deviation of the camera principal point position in the width direction in the camera coordinate system; Indicates the deviation of the camera's principal point position in the height direction in the camera coordinate system;

[0057] The error equation of bundle adjustment is:

[0058] ;

[0059] Among them, X1, X2, and X3 are the world coordinate correction values ​​of the intrinsic parameters, extrinsic parameters, and measurement points (marker points on the calibration plate involved in the calculation), respectively; A, B, and C are the partial derivative matrices corresponding to the intrinsic parameters, extrinsic parameters, and measurement point world coordinates, respectively; specifically expressed as:

[0060] ;

[0061] ;

[0062] Where V represents the camera residual; Represents the component of the camera residual on the X-axis of the image coordinate system; Represents the component of the camera residual on the Y axis of the image coordinate system; L represents the deviation between the observed real image point coordinates and the theoretical image point coordinate initial value; Indicates the initial value of the X coordinate of the image point obtained by actual measurement; Indicates the initial value of the X coordinate of the theoretical point image corresponding to the measured image point; Indicates the initial value of the Y coordinate of the image point obtained by actual measurement; Indicates the initial Y coordinate value of the theoretical point image corresponding to the measured image point.

[0063] The marking points of the dot matrix calibration plate are composed of regularly arranged circular coding points and non-coding points. The spacing between the marking points is fixed and the world coordinates of all the marking points on the calibration plate are known (this world coordinate system is the world coordinate system where the marking points are determined by photogrammetry, not the world coordinate system of the multi-camera measurement system established by the present invention).

[0064] The resection method described above establishes a unified error equation for all cameras, bundling and optimizing the intrinsic parameter values ​​of all cameras. This refers to taking the image point coordinates of the marker points on a calibration plate that are covered by multiple images collected by all cameras as observation values, where the certain number means that all cameras are required to collect calibration plate images at the same time, and the calibration plate images taken by each camera must contain at least 40% of the marker point images on the calibration plate. Given the world coordinates of all marker points on the calibration plate, and since each image corresponds to a total of 16 unknown camera intrinsic and extrinsic parameters, each image contains the image point coordinates of 8 marker points to solve the intrinsic and extrinsic parameter values ​​of the camera corresponding to the current image. After the calibration plate is moved to different positions, the same camera corresponds to multiple images, and the error equations of multiple images are combined to optimize and solve the optimal initial values ​​of the intrinsic and extrinsic parameters of all cameras.

[0065] It's important to note that the error equation is crucial for determining the camera's intrinsic and extrinsic parameters. Initial or observed values ​​must be provided before each calculation, allowing for bundle adjustment. If any intrinsic or extrinsic parameters, or the world coordinates of the measurement point, are known or not involved in the calculation, the corresponding correction is zero.

[0066] As an optimal technical solution, the method for solving the initial values ​​of the extrinsic parameters of the multi-camera measurement system is as follows: first, the center point of the dot matrix calibration plate is placed near the center point within the measurement format, and the point is set as the origin of the world coordinate system of the multi-camera measurement system; then, the cameras in the multi-camera measurement system with a common field of view reconstruct the marker points on the calibration plate that are located in the common field of view and can be photographed, and the back intersection method is applied to first solve the extrinsic parameter values ​​of the cameras participating in the reconstruction of the marker points; then, the calibration plate is removed from the measurement format, and coded marker points with non-repeated numbers are randomly placed, and the camera with existing extrinsic parameters is used to photograph and reconstruct the coordinates of the coded marker points that can be photographed; finally, the number ID matching method is used to determine other cameras that may participate in the reconstruction of the aforementioned marker points in addition to the aforementioned camera, and based on the cone method, combined with the existing marker point coordinates, the initial values ​​of the extrinsic parameters of the cameras that may participate in the reconstruction of the marker points are solved.

[0067] The world coordinate system of the multi-camera measurement system is a crucial prerequisite for dynamic calibration. While the relative positions of the cameras in the multi-camera measurement system may shift due to external factors (such as environmental vibration and camera self-heating), the world coordinate system at the center of the measurement format is anchored and does not move. The XOY plane of the world coordinate system lies within the measurement plane, with the Z axis pointing vertically upward, with the positive Z axis pointing toward the camera.

[0068] The marker points on the reconstruction calibration plate are used to solve the external parameters of the camera in the public field of view. The camera in the public field of view is used to synchronously capture the images of the marker points on the calibration plate within the field of view. The intrinsic parameters of each camera and the world coordinates of the marker points are known. The rear intersection method is applied to the cameras involved in reconstructing the marker points, and an error equation is uniformly established. The extrinsic parameter values ​​of the camera in the public field of view are obtained by bundling and adjusting.

[0069] The ID matching determines the camera that reconstructs the coded marker points and solves the initial values ​​of the extrinsic parameters of the cameras involved in the reconstruction of the marker points. It is required that the coded marker points with non-repeated IDs are randomly arranged within the measurement format, and the coded marker points cannot be stacked. Based on the camera with existing extrinsic parameters, the world coordinates of a portion of the coded marker points are first reconstructed; then, in all camera images of the multi-camera measurement system, the uniqueness of the coding ID is used to match and determine the cameras that participated in the reconstruction of the above-mentioned coded marker points, except for the camera with existing extrinsic parameters; since the number of reconstructed coded marker points is limited, solving the extrinsic parameters of the other cameras that participated in the reconstruction of the above-mentioned coded marker points (hereinafter referred to as "other cameras") is a special case of the resection method, called single-image resection. In this case, the intrinsic parameters of the camera and the world coordinates of the coded marker points are already known, and the corresponding error equation can be expressed as:

[0070] ;

[0071] Where, represents the camera residual; represents the partial derivative matrix corresponding to the extrinsic parameters; L represents the initial value deviation between the actual measured point image coordinates and the corresponding theoretical point image coordinates; The world coordinate correction number representing the extrinsic parameters.

[0072] Since the camera's extrinsic parameters only contain six unknowns, the calculation can be performed with only the world coordinates of the three coded markers. It is important to note that single-image resection is an iterative operation after linearizing a nonlinear equation. Initial values ​​for the camera's extrinsic parameters must be given before the calculation to ensure convergence and solvability of the final equation.

[0073] The method of solving the initial value of camera extrinsic parameters by single image resection is called the cone method. The extrinsic parameters corresponding to the camera image are determined by applying the principle that the vertex angle between the image space and the actual object space of the photographic beam cone is equal. Figure 3 :

[0074] In the figure, 、 、 are the coordinates of the three coded markers in the world coordinate system, 、 、 The three coded markers correspond to the image point coordinates of a camera S, S-xyz is the camera coordinate system, and O-XYZ is the world coordinate system. and S- Similar, because 、 ,as well as If they are unknown, we can use other cameras to capture the coded markers with known world coordinates, and combine them with the image coordinates corresponding to the markers captured by other cameras to solve them through multiple iterations. 、 ,as well as .

[0075] ;

[0076] ;

[0077] ;

[0078] At the same time, determine the marking point 、 、 Coordinates in the S-xyz camera coordinate system 、 、 , specifically expressed as:

[0079] ;

[0080] ;

[0081] ;

[0082] The initial value of the camera external parameters is to solve the matrix transformation relationship between each camera coordinate system and the world coordinate system. The specific solution process includes: first, solving the coordinates of the center of the three markers in the world coordinate system and the camera coordinate system 、 :

[0083] ;

[0084] ;

[0085] Apply the point coordinates of the center of the three marker points in the world coordinate system and the camera coordinate system 、 , combined with the point coordinates of the three marker points in the world coordinate system 、 、 And the point coordinates in the camera coordinate system 、 、 ;

[0086] Solve the rotation matrix from the world coordinate system to the camera coordinate system for:

[0087] ;

[0088] Continue to solve the translation matrix from the camera coordinate system to the world coordinate system for:

[0089] ;

[0090] in, Represents the coordinates of the center of gravity of the three landmarks in the camera coordinate system; Represents the coordinates of the centers of the three markers in the world coordinate system. So far, the rotation and translation matrices from the world coordinate system of other cameras to the camera coordinate system have been solved. 、 The transpose of the rotation and translation matrix is ​​the matrix transformation relationship from the camera coordinate system of other cameras to the world coordinate system, and is also the initial value of the external parameters of other cameras. 、 It should be noted that the above method for solving the initial values ​​of the extrinsic parameters of other cameras requires the world coordinates of at least three coded markers to be known.

[0091] As a preferred technical solution, the method for bundling and optimizing the extrinsic parameter values ​​of all cameras with existing initial extrinsic parameter values ​​includes: in view of the aforementioned technical solution, the intrinsic parameters of all cameras in the multi-camera measurement system and the initial extrinsic parameter values ​​of all cameras involved in reconstructing the coded markers are known, and the error equation of the bundled adjustment is simplified:

[0092] ; Among them, B and C are the partial derivative matrices corresponding to the external parameters and the world coordinates of the measurement point respectively;

[0093] In the optimization process, the centroid coordinates of all coded markers are kept unchanged as a constraint to ensure that the coordinate system does not shift. The specific method includes: first, taking 7 coded markers as observation values, since the coded markers are linearly independent of each other, the following constraint equation E is obtained according to the conditional least squares adjustment method:

[0094] ;

[0095] The physical meaning of this equation is that all encoded marker points, including the camera's extrinsic parameters, their center of mass and the sum of the vectors from all points to the center of mass remain unchanged, that is, all points are constrained to be anchored in one position and cannot undergo overall rotation.

[0096] Here, d represents the number of observation points, expressed as 7; u represents the number of unknowns, expressed as the camera extrinsic parameters to be solved and the world coordinates of the coded markers, expressed as 12. The first three rows of the equation are the derivatives of the three equations with respect to the unknowns, which are constant at the center of mass of all the coded markers. The middle three rows are the reciprocals of the three equations with respect to the unknowns, which are constant at the sum of the vectors from all points to the center of mass. The last row is the constraint that all 3D points are proportionally constant. The number of columns in the matrix represents the number of unknowns to be solved.

[0097] The above optimization algorithm first uses 7 coded marker points as observation values, and then uses this optimization method to use all coded marker points in the measurement format as observation values ​​until all coded marker points have participated in the above optimization algorithm.

[0098] Since the constraint of keeping the centroid coordinates of the coding mark points unchanged is added to the existing error equation, the Lagrange multiplier method is applied to add the augmented matrix of the constraint to the original error equation matrix and add a new coefficient matrix to the unknowns to solve the error equation with the constraint condition. The specific expression is:

[0099] ;

[0100] Represents the coefficients of the error equation for bundle adjustment, which contains two parts, B and C, which are the extrinsic parameters and the partial derivative matrix corresponding to the world coordinates of the measurement point respectively;

[0101] represents the coefficient of the aforementioned constraint equation. ;

[0102] represents the introduced Lagrangian factor;

[0103] Represents the camera external parameters obtained by optimization and the world coordinates of the measurement points, which include The two parts represent the external parameters of all cameras involved in reconstructing the encoded markers and the world coordinate correction number of the measurement point (the marker point randomly walked in the measurement plane).

[0104] The center of gravity of the coding mark point does not move, which means that when the cameras of the multi-camera measurement system capture the coding mark points in the measurement format, all the coding mark points remain stationary and do not move.

[0105] As a preferred technical solution, the method of looping the above two algorithm flows, solving the extrinsic parameter values ​​of all cameras in the multi-camera measurement system, and applying the proportional relationship of the point spacing of the ruler to correct and output the accurate extrinsic parameter values ​​of all cameras includes: first, after executing the above two algorithm flows for the first time, at least one camera with known extrinsic parameters can be added to at least the cameras with a common field of view in the multi-camera measurement system; then, using the cameras of the multi-camera measurement system with known extrinsic parameters, starting from the reconstructed coded marker points, diffuse outward to reconstruct other coded marker points within the measurement format, and again solve for the camera extrinsic parameters that may participate in the other coded marker points; then, continuing the above two algorithm flows until all coded marker points within the measurement format are reconstructed, that is, the extrinsic parameter values ​​of all cameras in the multi-camera measurement system are solved; finally, applying the proportional relationship of the point spacing of the ruler, the extrinsic parameter values ​​of all cameras in the multi-camera measurement system are corrected and output.

[0106] The aforementioned two algorithmic processes are looped until the extrinsic parameter values ​​for all cameras in the multi-camera measurement system are solved, resolving the problem of a multi-camera measurement system's limited shared field of view, which prevents the extrinsic parameters from being determined. Using the pyramid method, only a small number of coded markers need to be reconstructed, starting with the camera with a shared field of view and continuously expanding outward to determine the initial extrinsic parameter values ​​for the remaining cameras. By continuously looping through the algorithmic process, all coded markers within the measurement format are reconstructed, ultimately determining the extrinsic parameter values ​​for all cameras in the multi-camera measurement system.

[0107] The proportional relationship of the point distance of the applied ruler is used to correct and output the extrinsic parameter values ​​of all cameras in the multi-camera measurement system. First, the point distance measurement value of the marked points on the ruler is known. The multi-camera measurement system with known extrinsic parameter values ​​of all cameras is used to shoot the ruler and reconstruct the marked points on both sides of the ruler to determine the proportional relationship between the measurement point distance and the measurement point distance; then, with the constraint that the origin of the world coordinate system remains unchanged, the measurement point distance is corrected to the measurement point distance according to the above proportional relationship; finally, after the measurement point distance is corrected, the world coordinate system is also scaled accordingly, and more accurate extrinsic parameter values ​​of all cameras are output accordingly. 、 As the initial value of the external parameters of the camera system for subsequent initialization calibration. At this point, the calibration of the multi-camera measurement system is completed, and the internal and external parameter values ​​of all cameras in the multi-camera measurement system are determined.

[0108] As a preferred technical solution, the initialization calibration of the multi-camera measurement system includes: first, starting the backlight source within the measurement format, completing the multi-camera synchronous acquisition of all factory parameter calibration and reconstructing the control points of the outer circle of the measurement format, and reconstructing the world coordinates of the control points. Then, since the intrinsic parameters of the camera system are fixed and do not change, a simplified bundle adjustment error equation is established for all cameras:

[0109] ;

[0110] Where V represents the camera residual; B and C are the extrinsic parameters and the partial derivative matrices corresponding to the world coordinates of the measurement point, respectively; L represents the initial value deviation between the actual measured point image coordinates and the corresponding theoretical point image coordinates; The world coordinate correction number representing the extrinsic parameters; Indicates the world coordinate correction value of the measurement point.

[0111] The world coordinates of the applied control point And the extrinsic parameter values ​​of all cameras in the multi-camera measurement system determined during the calibration phase 、 As the initial value of the parameters involved in the adjustment, the above error equation is bundled and adjusted to obtain the accurate world coordinates of the control points of the outer circle of the measurement format And the extrinsic parameter values ​​of all cameras in the multi-camera measurement system after initialization and calibration 、 .

[0112] Finally, the accurate world coordinates of the control points after binding adjustment are obtained , solve the mean distance from all control points to the origin of the world coordinate system ; At the same time, record the ambient temperature during initial calibration , complete the initialization calibration.

[0113] As a preferred technical solution, the dynamic calibration of the multi-camera measurement system includes the following: First, in order to ensure the measurement accuracy under the long-term working environment of the multi-camera measurement system, the multi-camera measurement system needs to be dynamically calibrated before each measurement. First, the backlight within the measurement format is kept on, and the multi-camera measurement system synchronously collects and reconstructs the control points of the outer circle of the format to obtain the world coordinates of the control points. ; Then, apply the world coordinates of the reconstruction control points and the exact world coordinates of existing control points Determine whether the current measurement meets the measurement requirements; then, confirm the measurement temperature According to the principle of thermal expansion, the mean distance between all control points and the world origin measured during the dynamic calibration phase is corrected. The theoretical point distance mean ; Finally, apply the corrected mean distance , and with the world coordinate system origin always in motion as the constraint condition, dynamically calibrate the external parameters of the multi-camera measurement system for this measurement 、 .

[0114] The control points are circular marking points distributed around the periphery of the measurement format and passively illuminated by the bottom light source. They have clear edges, obvious black and white contrast, and are easy to identify.

[0115] The application reconstructs the world coordinates of the control points and the exact world coordinates of existing control points Methods for determining whether the current measurement meets the measurement requirements include: First, the world coordinates and world coordinates Located in the same world coordinate system, based on the least squares theory, the method of applying the minimum distance to identify the same-name points is used to match and determine and The calibration is then validated based on the matching results. If no more than three control points fail to match or are invalid, the calibration is valid; otherwise, the calibration is invalid and the process is discontinued. It is important to note that control point matching can filter out invalid calibrations caused by obstacles, missing control points, and excessive camera offsets.

[0116] The confirmed measured temperature , apply the principle of thermal expansion to correct the control point distance in the diagonal direction of the measurement area Control point distance The method specifically includes: first, after measuring and determining, recording the measured ambient temperature , combined with the ambient temperature during initial calibration , determine the measured temperature difference :

[0117] ;

[0118] Then, according to the principle of thermal expansion, the theoretical mean distance between all control points and the origin of the world coordinate system under long-term light illumination can be solved: :

[0119] ;

[0120] in, Indicates the number of bundle adjustments, Indicates the During the first real-time calibration, the length change caused by the thermal expansion principle, Indicates the thermal expansion coefficient of the control point material.

[0121] Then, according to the reconstructed world coordinates of the control points , solve the mean distance from all control points to the world origin measured by the real-time multi-camera measurement system ;

[0122] Finally, solve the mean distance of the actual measurement Mean distance between control points and ideal measurement environment under thermal expansion The proportional factor is used to correct the mean value of the measured point distance. The theoretical point distance mean .

[0123] The control point distance from the mean after the application correction is , and with the world coordinate system origin always in motion as the constraint condition, dynamically calibrate the external parameters of the multi-camera measurement system for this measurement 、 The specific method includes: First, since the internal parameters of all cameras are fixed and remain unchanged during the calibration process, the point distance mean is applied , calibrate the displacement of the control points caused by thermal expansion during measurement; then, with the world coordinate system origin always being fixed as a constraint, combine the accurate world coordinates of the control points on the outer circle of the measurement format And the extrinsic parameter values ​​of all cameras in the multi-camera measurement system after initialization and calibration 、 As the initial value of the bundle adjustment parameters, the simplified bundle adjustment error equation is applied to all cameras:

[0124] ;

[0125] Bundled adjustment to obtain the external parameters of all cameras in the multi-camera measurement system after real-time calibration 、 .in, Indicates the number of bundle adjustments.

[0126] It should be noted that the initial calibration process is based on the initial external parameters of the multi-camera measurement system determined by system calibration. 、 and the world coordinates of the control points reconstructed during the initial calibration phase The initial value of the parameters for bundle adjustment; the dynamic calibration process is to initialize the external parameters of the multi-camera measurement system 、 And the world coordinates of the global point after initialization calibration Using the world coordinate system origin as the initial value for the bundle adjustment parameters, while also constraining the bundle adjustment to the fact that the world coordinate system origin remains stationary, ensures that the extrinsic parameter relationships of the multi-camera measurement system are not affected by external factors and shifted. While maintaining measurement accuracy, the dynamically calibrated extrinsic parameters of the multi-camera measurement system are more consistent with the actual measurement environment.

[0127] like Figure 2 The measurement system shown includes: a box 9, a camera 1, a tempered glass 3, a backlight source 2 and a controller 7;

[0128] A plurality of cameras 1 are arranged at intervals around the top inner edge of the box 9; the camera located in the middle area of ​​the box edge is the central area camera 8;

[0129] The backlight source 2 is provided on the bottom end surface of the tempered glass 3, and the coding mark point 4 and the backlight mark point 6 are provided on the top end surface of the tempered glass 3;

[0130] The tempered glass 3 is placed on the bottom end surface of the box 9, and a marking point scale 5 is provided on the top end surface of the tempered glass 3;

[0131] The controller 7 is connected to the cameras 1 and backlight source 2, respectively. All cameras 1 in the multi-camera measurement system synchronously capture and reconstruct backlight markers 6 fixed on the plane of the tempered glass 3. Simultaneously, the multi-camera measurement system calculates and corrects the offset between control points caused by thermal expansion. Combined with the camera extrinsic parameters determined during the system initialization and calibration phase, this completes the system's real-time calibration. This process requires no human intervention, with acquisition and calculation taking no more than 2 seconds. This overcomes the effects of environmental vibration, camera self-heating, and system inaccuracy on the measurement accuracy of the multi-camera measurement system, truly achieving low-cost, high-efficiency, and high-precision real-time self-calibration for the multi-camera measurement system.

[0132] In addition, the present invention also provides an optimized calibration method for the calibration function of a multi-camera measurement system, which is undergoing factory full parameter calibration. Through factory full parameter calibration, a world coordinate system is established at the center point of the measurement area. Using the cone method, the extrinsic parameters of the cameras in the public field of view in the central area and the world coordinates of the coded markers in the public field of view are known, and the initial values ​​of the camera extrinsic parameters of some other cameras are continuously diffused outward to solve. At the same time, the initial values ​​of the extrinsic parameters of all currently known cameras are bundled and adjusted to obtain the optimized extrinsic parameter values ​​of all cameras involved in the reconstruction of the coded markers. By analogy, the optimized extrinsic parameter values ​​of all cameras in the multi-camera measurement system are obtained.

[0133] Based on the above-mentioned optimized calibration method, after completing the last extrinsic parameter solution and bundling adjustment, a ruler with a known marking point distance is introduced. The proportional relationship between the measurement point distance and the metering point distance is used to constrain and correct the scaling of the world coordinate system, outputting more accurate extrinsic parameters of all cameras.

[0134] Among them, the inventive point of this application is the factory full parameter calibration: technical optimization is carried out on the basis of the existing calibration algorithm, breaking its technical limitations and solving the internal and external parameter values ​​of the multi-camera measurement system.

[0135] Intrinsic parameter calibration: breaks the existing calibration algorithm's requirements for the calibration plate size.

[0136] A dot matrix calibration plate printed with coded and non-coded markers is used, and multiple positions of the calibration plate are placed at various positions of the measurement format. The world coordinates of all markers on the calibration plate are known. Combined with the images of the markers captured by the camera, error equations are established for all cameras based on the rear intersection method, and the intrinsic parameter values ​​of each camera are uniformly optimized and adjusted.

[0137] Extrinsic parameter calibration: Breaking the public field of view requirement of existing calibration algorithms for cameras.

[0138] The calibration plate is placed at the center of the measurement format, the world coordinate system is established, and the extrinsic parameter values ​​of some cameras in the central area are solved; then, the calibration plate is removed and coded markers with non-repeated numbers and a marker scale with known point spacing are randomly arranged. By matching the coded IDs, all cameras involved in the reconstruction of each coded marker are determined; based on the cone method, the initial values ​​of the extrinsic parameters of other cameras involved in the reconstruction of the coded markers, except for the camera with known extrinsic parameters, are calculated; for all the aforementioned cameras, a unified error equation is established, and the extrinsic parameter values ​​of all cameras are bundled and optimized; since the coded markers arranged in the measurement format have already covered the measurement format, the bd steps are continuously repeated according to the number of cameras in the multi-camera measurement system until the extrinsic parameter values ​​of all cameras in the multi-camera measurement system are obtained; to ensure that the measurement results conform to the actual physical dimensions, a scale with known point spacing is introduced. After the cyclic measurement is completed, the point spacing ratio on the scale is applied to correct and output the accurate extrinsic parameter values ​​of all cameras in the multi-camera measurement system.

[0139] On-site calibration: On-site calibration innovatively introduces a method of reconstructing control points, solving the problem of unguaranteed measurement accuracy of multi-camera measurement systems during actual measurement processes.

[0140] Initialization calibration: Introduce initialization measurement parameters for dynamic calibration to ensure calibration accuracy.

[0141] Using a calibrated multi-camera measurement system, the control points within the measurement format are reconstructed for the first time. This calibration results in a set of extrinsic parameter relationships for the multi-camera measurement system. This set of extrinsic parameter relationships will serve as the initial values ​​for the extrinsic parameters used for dynamic calibration of the multi-camera measurement system. Simultaneously, the ambient temperature and the mean distance from all control points to the origin of the world coordinate system are recorded.

[0142] Dynamic calibration: Before each measurement, the multi-camera measurement system reconstructs the control points to determine whether to perform dynamic calibration.

[0143] Based on the control point data obtained from the initialization calibration, the multi-camera measurement system synchronously collects and reconstructs the control points within the measurement format, anchors the world coordinate system, that is, the origin of the world coordinate system remains stationary, and applies the principle of thermal expansion to dynamically calibrate the external parameter relationships of the multi-camera measurement system.

[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A self-calibration method for a multi-camera measurement system, characterized in that: The method comprises the following steps: Step 1: Intrinsic parameter optimization: Place a calibration plate with coded and non-coded markers within the measurement field of view. Use multiple cameras to synchronously capture images of the calibration plate in different poses. Establish an error equation based on the resection method and perform global optimization to obtain the intrinsic parameter values ​​of each camera. Step 2: Solve the initial values ​​of extrinsic parameters: Use the center point of the calibration plate as the world coordinate origin and reconstruct the initial extrinsic parameters by using the marker points of the cameras in the public field of view. After removing the calibration plate, arrange the coded marker points and use the existing extrinsic parameter camera to reconstruct the marker point coordinates. Combined with the cone method, solve the initial values ​​of the extrinsic parameters of other cameras. The initial value solution of the external parameters includes: placing the center point of the dot matrix calibration plate near the center point of the measurement format, and setting the point as the origin of the world coordinate system of the multi-camera measurement system; the point is the point where the center point of the dot matrix calibration plate coincides with the measurement format; In a multi-camera measurement system, for cameras with a common field of view, the camera reconstructs the markers on the calibration plate that are located in the common field of view and can be photographed. The resection method is then used to first solve the camera extrinsic parameters involved in the reconstruction of the markers. Remove the calibration plate from the measurement area and randomly place non-repeated coded markers. Use a camera with known extrinsic parameters to capture and reconstruct the coordinates of the captured coded markers. Using the ID matching method, determine the cameras other than the camera with existing extrinsic parameters that participate in the aforementioned marker reconstruction. Based on the cone method and combined with the existing marker coordinates, solve the initial values ​​of the camera extrinsic parameters participating in the marker reconstruction. The marking points of the dot matrix calibration plate are composed of regularly arranged circular coding points and non-coding points. The spacing between the marking points is fixed and the world coordinates of all the marking points on the calibration plate are known. Step 3: Optimize the external parameters: Establish the error equations of all cameras, perform bundle optimization with the constant coordinates of the center of gravity of the marker points as the constraint condition, and obtain accurate external parameters; Step 4: Scale correction: perform proportional correction on the optimized external parameters through the scale point spacing ratio; Step 5: Dynamic calibration: Before each measurement, control points are collected synchronously, and the relationship between external parameters is dynamically calibrated based on the principle of thermal expansion in combination with ambient temperature data.

2. The self-calibration method for a multi-camera measurement system according to claim 1, wherein: The intrinsic parameter optimization described in step 1 involves using a dot matrix calibration plate with markers, continuously moving the plate within the measurement format, and placing it in different poses at each position. Simultaneously, all cameras synchronously capture images of the calibration plate. Then, based on the resection method, a unified error equation is established for all cameras, which are then bundled and optimized to obtain the intrinsic parameter values ​​for all cameras. The intrinsic parameter values ​​include: lens focal length , lens distortion 、 、 、 、 、 、 , principal point deviation 、 ; Among them, lens distortion refers to the lens distortion caused by the lens processing technology and the lens assembly position; 、 、 represents the mirror distortion parameter; 、 represents the eccentricity distortion parameter; 、 represents the plane distortion parameter; The principal point deviation indicates the deviation of the camera principal point in the camera coordinate system. Indicates the deviation of the camera principal point in the width direction in the camera coordinate system; Indicates the deviation of the camera's principal point position in the height direction in the camera coordinate system.

3. The self-calibration method for a multi-camera measurement system according to claim 1, wherein: The external parameter values ​​include: , represents the matrix transformation relationship from the camera coordinate system to the world coordinate system; R represents the rotation matrix relationship from the camera coordinate system to the world coordinate system; T represents the translation matrix relationship from the camera coordinate system to the world coordinate system; Based on the intrinsic and extrinsic parameter values, the error equation is: Where X1, X2, and X3 are the intrinsic parameters, extrinsic parameters, and the world coordinate correction values ​​of the measurement point, respectively; A, B, and C are the partial derivative matrices corresponding to the intrinsic parameters, extrinsic parameters, and the world coordinates of the measurement point, respectively; L represents the deviation between the observed real image point coordinates and the theoretical image point coordinate initial values. The corresponding formula is: ； ； Where V represents the camera residual; Represents the component of the camera residual on the X-axis of the image coordinate system; Represents the component of the camera residual on the Y axis of the image coordinate system; Indicates the initial value of the X coordinate of the image point obtained by actual measurement; Indicates the initial value of the X coordinate of the theoretical point image corresponding to the measured image point; Indicates the initial value of the Y coordinate of the image point obtained by actual measurement; Indicates the initial Y coordinate value of the theoretical point image corresponding to the measured image point.

4. The self-calibration method for a multi-camera measurement system according to claim 1, wherein: The resection method is to uniformly establish error equations for all cameras, bundle and optimize to obtain the intrinsic parameter values ​​of all cameras; The coordinates of the image points of the marker points on the calibration plate that cover a certain number of images collected by all cameras are used as observation values. The certain number means that all cameras are required to collect calibration plate images at the same time, and the calibration plate images taken by each camera must contain at least 40% of the marker points on the calibration plate. Given the world coordinates of all marker points on the calibration plate, and since each image corresponds to a total of 16 unknown camera intrinsic and extrinsic parameters, the intrinsic and extrinsic parameters of the camera corresponding to the current image can be solved by using the coordinates of the image points of 8 marker points in each image. Among them, after the calibration plate moves to different positions, the same camera corresponds to multiple images, and the error equations of multiple images are combined to optimize and solve the optimal initial values ​​of the internal and external parameters of all cameras.

5. The self-calibration method for a multi-camera measurement system according to claim 1, wherein: The ID matching determines the camera for reconstructing the coded marker points, and when solving the initial values ​​of the external parameters of the cameras involved in reconstructing the marker points, the coded marker points with non-repeated IDs are randomly arranged within the measurement format; The process of solving the initial values ​​of the camera extrinsic parameters involved in the reconstruction of the marker points includes: firstly reconstructing the world coordinates of a portion of the encoded marker points based on the camera with the existing extrinsic parameter values; Continue to match and determine the cameras that participate in reconstructing the coded markers in all camera images, except for the cameras with existing external parameter values, based on the uniqueness of the coded ID; The single image resection method is used to calculate the initial values ​​of the external parameters of other cameras involved in reconstructing the coded markers. Since the intrinsic parameters of the cameras and the world coordinates of the coded markers are known, the corresponding error equation is expressed as: ； Where V represents the camera residual; B represents the partial derivative matrix corresponding to the external parameters; L The deviation between the actual measured point image coordinates and the initial value of the corresponding theoretical point image coordinates; The world coordinate correction number representing the extrinsic parameters.

6. The self-calibration method for a multi-camera measurement system according to claim 5, wherein: The method for solving the initial values ​​of camera extrinsic parameters by single image resection is the cone method, that is, the extrinsic parameters corresponding to the camera image are determined by applying the principle that the vertex angles between the image space and the actual object space of the photographic beam cone are equal.

7. A multi-camera measurement system, applied to the calibration method according to any one of claims 1 to 6, characterized in that: The measuring system includes: a box, a camera, tempered glass, a backlight source and a controller; A plurality of cameras are arranged at intervals around the top inner edge of the box; The backlight source is provided on the bottom end surface of the tempered glass, and the coding mark point and the backlight mark point are provided on the top end surface of the tempered glass; The tempered glass is placed on the bottom end surface of the box body, and a marking point scale is provided on the top end surface of the tempered glass; The controller is connected to the camera and the backlight source respectively.

Images