Self-calibration method and system of multi-camera measurement system

Abstract

The invention discloses a self-calibration method and system for a multi-camera measurement system, and belongs to the technical field of camera calibration, and the method comprises the steps: arranging a calibration plate with coded mark points and non-coded mark points in a measurement visual field, collecting different pose images of the calibration plate, building an error equation, carrying out the global optimization, and obtaining the internal reference value of each camera; taking the center point of the calibration plate as a world coordinate origin, and obtaining initial external parameters through mark point reconstruction of a camera in a public view; after the calibration plate is removed, code mark points are arranged, the coordinates of the mark points are reconstructed through existing external parameter cameras, and external parameter initial values of other cameras are solved; establishing error equations of all cameras, and obtaining accurate external parameters by taking the barycentric coordinates of the mark points as constraint conditions; according to the method, the coding mark points and the mark point scales are applied, the problems that the external parameters of other cameras without a common view field are difficult to solve and the operation cost is high are solved, and meanwhile accurate external parameter initial values of all the cameras are provided for subsequent calibration.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and specifically to a self-calibration method and system for a multi-camera measurement system. Background Art

[0002] Camera calibration is a key step in visual measurement and is used to determine the internal and external parameters of a camera. Among them, the internal parameters of a camera include: focal length, distortion, and principal point deviation; the external parameters of a camera include: the rotation and translation matrix relationship between the camera coordinate system and the world coordinate system.

[0003] Currently, the commonly used methods for calibrating the internal and external parameters of a camera in the industry include: First, place a calibration board within the common field of view of all cameras, and the true world coordinates of the feature points on the calibration board are known; then, all cameras synchronously collect and reconstruct the feature points; finally, establish a matrix equation between the true world coordinates of the feature points and the corresponding image coordinates captured by the cameras, and solve the internal and external parameters of each camera.

[0004] The calibration method has the following disadvantages: Technical limitations: The existing technical limitations are mainly reflected in the following two aspects. First, it highly depends on the format of the calibration board and the common field of view of the camera: The existing calibration technology requires cameras with a common field of view to capture a calibration board that matches the measurement format, so as to solve the internal and external parameters of the camera. Such strict operation requirements limit the actual measurement scenarios of the existing calibration methods. Second, repeated calibration causes coordinate system conversion problems: When using the existing technology for continuous calibration multiple times, it is easy to have the phenomenon that the origin of the coordinate system obtained from multiple calibrations is not unified. Especially for complex automated measurement systems, multi-coordinate system unified positioning is required for each measurement, which is very time-consuming and difficult. Environmental impact: Calibrating a multi-camera measurement system is inevitably affected by external factors. Specifically, 1) Vibration impact: In the measurement environment of a multi-camera measurement system, environmental vibration inevitably exists. The environmental vibration is transmitted through the structural components that fix the camera, resulting in changes in the external parameter relationship of the camera. Existing anti-vibration solutions (such as using materials with stronger anti-vibration capabilities) can alleviate the impact of environmental factors to a certain extent, but the hardware cost is too high and the engineering applicability is not strong. 2) Thermal expansion and contraction effect of materials: During the long-term measurement process, the heat generated by the camera is conducted to the metal structural components that fix the camera, resulting in thermal expansion deformation of the structural components, thereby changing the external parameter relationship of the camera. Selecting materials with a lower coefficient of linear expansion as structural components can reduce the degree of thermal expansion deformation, but it cannot completely avoid thermal expansion deformation and will increase the hardware cost of the system. Self-precision attenuation: When using a multi-camera measurement system for a long time, the rigidity of the structural component materials and the internal stress of the materials in the system will affect the external parameter relationship of the camera, thereby reducing its own measurement precision. The specific manifestations are as follows: First, insufficient material rigidity: Most of the materials used to stabilize the structure of the multi-camera measurement system are rigid materials (usually steel frames, aluminum alloys, iron frames, etc.). The larger the measurement field of view, the higher the requirement for material rigidity. At present, it is difficult to find materials that meet the rigidity requirements in large-field-of-view measurements. Second, self-stress release: After the multi-camera measurement system runs for a long time, the rigid components in the system will deform to varying degrees due to the continuous action of their own gravity and internal stress, resulting in changes in the external parameter relationship of the camera. This stress deformation comes from the material itself and is difficult to completely eliminate. Summary of the Invention

[0005] On the one hand, to solve the problems of the existing technology, the present invention provides a self-calibration method for a multi-camera measurement system, and the method includes the following steps: Step 1, internal parameter optimization: Arrange a calibration board with coded and non-coded marker points within the measurement field of view. Synchronously collect images of the calibration board in different poses by multiple cameras, establish an error equation based on the resection method and perform global optimization to obtain the internal parameter values of each camera; Step 2, Initial solution of external parameters: Take the center point of the calibration board as the origin of the world coordinate system, and obtain the initial external parameters through the reconstruction of the marker points of the cameras within the common field of view; After removing the calibration board, arrange coded marker points, use the existing external parameter cameras to reconstruct the coordinates of the marker points, and combine the corner cube method to match and solve the initial external parameters of other cameras; Step 3, Optimization of external parameters: Establish the error equations of all cameras, and perform bundle optimization with the constraint that the centroid coordinates of the marker points remain unchanged to obtain accurate external parameters; Step 4, Scale correction: Perform proportional correction on the optimized external parameters through the scale ratio of the scale points; Step 5, Dynamic calibration: Synchronously collect control points before each measurement, and combine the environmental temperature data to dynamically calibrate the external parameter relationship based on the principle of thermal expansion.

[0006] Further, the internal parameter optimization method in Step 1 includes: Using a dot matrix calibration board of marker points, continuously move the position of the calibration board within the measurement area, and place it in different poses at each position. At the same time, all cameras synchronously collect the calibration board images, and then based on the resection method, establish error equations for all cameras uniformly, and then bundle and optimize to obtain the internal parameter values of all cameras; The internal parameter values include: lens focal length , lens distortion ( , , , , , , ), principal point deviation ( , ); Lens distortion represents the lens distortion caused by the influence of lens processing technology and lens assembly position; ( ) represents the mirror distortion parameter; ( ) represents the decentering distortion parameter; ( ) represents the planar distortion parameter; Principal point deviation represents the deviation of the camera principal point in the camera coordinate system, represents the deviation of the camera principal point position in the width direction of the camera coordinate system; represents the deviation of the camera principal point position in the height direction of the camera coordinate system.

[0007] Further, in Step 2, the initial solution of the external parameters includes: Place the center point of the dot matrix calibration board near the center point within the measurement area, and set this point as the origin of the world coordinate system of the multi-camera measurement system; This point is the coincidence point of the center point of the dot matrix calibration board and the measurement area; In a multi-camera measurement system, there are cameras with a common field of view. Marking points on a calibration board located within the common field of view and that can be photographed are reconstructed, and the resection method is applied to solve for the external parameter values of the cameras participating in the reconstruction of the marking points for the first time. Remove the calibration board from the measurement area and randomly place non-repeating numbered coded marking points. Use the cameras with known external parameters to photograph and reconstruct the coordinates of the coded marking points that can be photographed. Using the method of matching by ID number, determine the other cameras that may participate in the reconstruction of the aforementioned marking points in addition to the cameras with known external parameters. Based on the pyramid method, combined with the existing marking point coordinates, solve for the initial values of the external parameters of the cameras that may participate in the reconstruction of the marking points. The marking points of the dot matrix calibration board are composed of regularly arranged circular coded points and non-coded points. The spacing between the marking points is fixed and the world coordinates of all the marking points on the calibration board are known.

[0008] Furthermore, the external parameter values include: (R, T), which represent the matrix transformation relationship from the camera coordinate system to the world coordinate system. R represents the rotation matrix relationship from the camera coordinate system to the world coordinate system; T represents the translation matrix relationship from the camera coordinate system to the world coordinate system. Based on the internal parameter values and the external parameter values, the error equation is: ; where X1, X2, and X3 are the corrections of the internal parameters, external parameters, and the world coordinates of the measurement points respectively; A, B, and C are the partial derivative matrices corresponding to the internal parameters, external parameters, and the world coordinates of the measurement points respectively; L represents the deviation between the observed true image point coordinates and the initial values of the theoretical image point coordinates. The corresponding formula is: ; ; In the formula, V represents the residual of the camera. represents the component of the camera residual on the X-axis of the image coordinate system; represents the component of the camera residual on the Y-axis of the image coordinate system; represents the initial value of the X coordinate of the image point obtained by actual measurement; represents the initial value of the theoretical point image X coordinate corresponding to the actually measured image point; represents the initial value of the Y coordinate of the image point obtained by actual measurement; represents the initial value of the theoretical point image Y coordinate corresponding to the actually measured image point.

[0009] Furthermore, the resection method is to uniformly establish an error equation for all cameras, and bundle and optimize to obtain the internal parameter values of all cameras. The image point coordinates of the marker points on the calibration plate that are collected by all cameras and cover a certain number of images are taken as observation values, where a certain number means that all cameras are required to collect images of the calibration plate at the same time, and the images of the calibration plate taken by each camera contain at least 40% of the marker point images on the calibration plate; the world coordinates of all the marker points on the calibration plate are known, and since the camera intrinsic and extrinsic parameters corresponding to each image have a total of 16 unknowns, the image point coordinates of 8 marker points in each image can be used to solve the intrinsic and extrinsic parameter values of the camera corresponding to the current image; Among them, after the calibration plate is moved to different positions, the same camera corresponds to multiple images, and the error equations of multiple images are combined to optimize and solve the optimal initial values of the internal and external parameters of all cameras.

[0010] Furthermore, when the ID matching determines the camera for reconstructing the coded mark points and solves the initial values of the external parameters of the camera involved in reconstructing the mark points, the coded mark points with non-repeated IDs are randomly arranged within the measurement format; The process of solving the initial values of the camera extrinsic parameters that may be involved in the reconstruction of the marker points includes: based on the camera with existing extrinsic parameter values, firstly reconstructing the world coordinates of a part of the encoded marker points; Continue to match and determine other cameras that may participate in reconstructing the coded marker points in all camera images, except for the cameras with existing external parameter values, through the uniqueness of the coded ID; The single image resection method can be used to calculate the initial values of the external parameters of other cameras that may be involved in reconstructing the coded markers. Since the intrinsic parameters of the camera and the world coordinates of the coded markers are known, the corresponding error equation can be expressed as: ; Where V represents the residual error of the camera; B represents the partial derivative matrix corresponding to the external parameters; L represents the initial value deviation between the actual measured point image coordinates and the corresponding theoretical point image coordinates; The world coordinate correction number representing the extrinsic parameters; Furthermore, the method for solving the initial value of the camera extrinsic parameters by single image back intersection is the cone method, which determines the extrinsic parameters corresponding to the camera image by applying the principle that the vertex angles between the image space and the light rays in the actual object space of the photographic beam cone are equal.

[0011] On the other hand, the present application provides a multi-camera measurement system, the measurement system comprising: a box, a camera, tempered glass, a backlight source and a controller; A plurality of cameras are arranged at intervals around the top inner edge of the box; The backlight source is arranged on the bottom end surface of the tempered glass, and the coding mark point and the backlight mark point are arranged on the top end surface of the tempered glass; The tempered glass is placed on the bottom end face inside the box body, and a marking point scale is arranged on the top end face of the tempered glass; The controller is respectively connected with the camera and the backlight source.

[0012] Advantages of the present invention: The present invention applies coded marking points and a marking point scale, overcomes the problems of difficult solution of external parameters of other cameras without a common field of view and high operation cost, and at the same time provides more accurate initial values of external parameters of all cameras in a multi-camera measurement system for subsequent calibration; by reconstructing the backlight marking points on the measurement plane through the multi-camera measurement system, the problem of loss of system measurement accuracy caused by environmental vibration and thermal expansion in the multi-camera measurement system is dynamically calibrated. Description of the drawings

[0013] Figure 1 It is a schematic diagram of the calibration process of the multi-camera measurement system provided by the present invention; Figure 2 It is a schematic diagram of the structure of the measurement system provided by the present invention; Figure 3 It is an image schematic diagram of the photographic light beam pyramid provided by the present invention.

[0014] Reference numerals: In the figure: 1 is a camera, 2 is a backlight source, 3 is a tempered glass, 4 is a coded marking point, 5 is a marking point scale, 6 is a backlight marking point, 7 is a controller, 8 is a central area camera, and 9 is a box body. Detailed implementation manners

[0015] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0016] Please refer to Figures 1 - 3 , the present invention provides a self-calibration method for a multi-camera measurement system, including: a calibration board printed with coded marking points and non-coded point marking points, placing the calibration board in multiple poses at various positions within the measurement field of view, and synchronously collecting images by each camera; based on the resection method, establishing error equations for all cameras and bundling and optimizing the internal parameter values of all cameras.

[0017] Place the center point of the dot matrix calibration board near the center point within the measurement range, and set this point as the world coordinate origin of the multi-camera measurement system; this point is the coincidence point of the center point of the dot matrix calibration board and the measurement range. At the same time, the cameras with a common field of view in the multi-camera measurement system reconstruct the marked points on the calibration board, and apply the resection method to solve the external parameter values of the cameras participating in the reconstruction of the marked points. Remove the calibration board and arrange non-repeating numbered coded marked points within the measurement range. Use the cameras with the existing external parameters to photograph and reconstruct the coordinates of the visible marked points. Use the method of ID matching to determine the other cameras that may participate in the reconstruction of the aforementioned marked points in addition to the aforementioned cameras. Based on the pyramid method, combined with the existing marked point coordinates, solve the initial values of the external parameters of the cameras that may participate in the reconstruction of the marked points.

[0018] Based on all the cameras with the initial values of the existing external parameters, establish an error equation uniformly and bundle and optimize the external parameter values of the cameras. During the optimization process, the centroid coordinates of all the marked points remain unchanged as a constraint to ensure that the coordinate system does not shift.

[0019] Loop the above two algorithm processes until the external parameter values of all the cameras in the multi-camera measurement system are solved. At the same time, apply the point distance ratio on the scale to correct and output the external parameter values of all the cameras in the multi-camera measurement system again.

[0020] Use the calibrated multi-camera measurement system to reconstruct the control points within the measurement range, record the ambient temperature and the average value of the distances from all the control points to the origin of the world coordinate system, and initialize and calibrate the external parameter relationship of the multi-camera measurement system.

[0021] Before each measurement, based on the measurement data of the initial calibration, all the cameras in the multi-camera measurement system synchronously collect and reconstruct the control points within the measurement range again, anchor the world coordinate system, that is, the origin of the world coordinate system remains unchanged, and apply the principle of thermal expansion to dynamically calibrate the external parameter relationship of the multi-camera measurement system.

[0022] As a preferred technical solution, the method for calibrating the internal parameters of the multi-camera measurement system includes: First, use the dot matrix calibration board of the marked points, continuously move the position of the calibration board within the measurement range, and place it in different poses at each position. At the same time, all the cameras in the multi-camera measurement system synchronously collect the calibration board images. Then, based on the resection method, establish an error equation uniformly for all the cameras, and bundle and optimize to obtain the internal parameter values of all the cameras. The internal parameters include: lens focal length ( ), lens distortion ( , , , , , , ), principal point deviation ( , ); The external parameters include: ( ), representing the matrix transformation relationship from the camera coordinate system to the world coordinate system; the principal point deviation ( , ); Lens distortion refers to the lens distortion caused by the influence of lens processing technology and lens assembly position; ( ) represents the mirror distortion parameter; ( ) represents the decentering distortion parameter; ( ) represents the planar distortion parameter; T represents the translation matrix relationship from the camera coordinate system to the world coordinate system; R represents the rotation matrix relationship from the camera coordinate system to the world coordinate system; the principal point deviation represents the deviation of the camera principal point in the camera coordinate system, represents the deviation of the camera principal point position in the width direction in the camera coordinate system; represents the deviation of the camera principal point position in the height direction in the camera coordinate system; The error equation for bundle adjustment is: ; Among them, X1, X2, and X3 are the corrections of the internal parameters, external parameters, and the world coordinates of the measurement points (the marked points on the calibration board participating in the calculation) respectively; A, B, and C are the partial derivative matrices corresponding to the internal parameters, external parameters, and the world coordinates of the measurement points respectively; the specific expressions are: ; ; In the formula, V represents the residual of the camera; represents the component of the camera residual on the X-axis of the image coordinate system; represents the component of the camera residual on the Y-axis of the image coordinate system; L represents the deviation between the observed true image point coordinates and the initial values of the theoretical image point coordinates; represents the initial value of the X coordinate of the image point obtained by actual measurement; represents the initial value of the theoretical image X coordinate of the theoretical point corresponding to the measured image point; represents the initial value of the Y coordinate of the image point obtained by actual measurement; represents the initial value of the theoretical image Y coordinate of the theoretical point corresponding to the measured image point.

[0023] The marked points of the dot matrix calibration board are composed of regularly arranged circular coded points and non-coded points, the spacing of the marked points is fixed and the world coordinates of all the marked points on the calibration board are known (this world coordinate system is the world coordinate system determined by photogrammetry for the marked points, rather than the world coordinate system of the multi-camera measurement system established in the present invention).

[0024] The described rear intersection method uniformly establishes error equations for all cameras, and bundles and optimizes the intrinsic parameter values of all cameras. It refers to taking the image point coordinates of the marker points on the calibration plate that are collected by all cameras in multiple images covering a certain number as observation values, where a certain number means that all cameras are required to collect calibration plate images at the same time, and the calibration plate images taken by each camera contain at least 40% of the marker point images on the calibration plate; the world coordinates of all marker points on the calibration plate are known, and since the camera intrinsic and extrinsic parameters corresponding to each image total 16 unknowns, each image contains the image point coordinates of 8 marker points to solve the intrinsic and extrinsic parameter values of the camera corresponding to the current image. After the calibration plate is moved to different positions, the same camera corresponds to multiple images, and the error equations of multiple images are combined to optimize and solve the optimal initial values of the intrinsic and extrinsic parameters of all cameras.

[0025] It should be noted that the error equation is an important equation for solving the intrinsic and extrinsic parameters of the camera. Before each calculation, the initial value or observed value of the parameter must be given before the bundle adjustment calculation can be performed. Among them, if there are intrinsic parameters or extrinsic parameters or the world coordinates of the measurement point are known or do not participate in the calculation, the corresponding correction number is zero.

[0026] As an optimal technical solution, the method for solving the initial values of the extrinsic parameters of the multi-camera measurement system is: first, the center point of the dot matrix calibration plate is placed near the center point in the measurement format, and the point is set as the origin of the world coordinate system of the multi-camera measurement system; then, the cameras in the multi-camera measurement system with a common field of view reconstruct the marking points on the calibration plate that are located in the common field of view and can be photographed, and the rear intersection method is applied to first solve the camera extrinsic parameter values participating in the reconstruction of the marking points; then, the calibration plate is withdrawn from the measurement format, and coded marking points with non-repeated numbers are randomly placed, and the camera with existing extrinsic parameters is used to photograph and reconstruct the coordinates of the coded marking points that can be photographed; finally, the number ID matching method is used to determine the cameras other than the aforementioned cameras that may participate in the reconstruction of the aforementioned marking points, and based on the cone method, combined with the existing marking point coordinates, the initial values of the camera extrinsic parameters that may participate in the reconstruction of the marking points are solved.

[0027] The world coordinate system of the multi-camera measurement system is an important condition for the multi-camera measurement system to achieve dynamic calibration. Affected by external factors (environmental vibration, camera self-heating, etc.), the relative position of the camera of the multi-camera measurement system may move, but the world coordinate system of the center point of the measurement format is anchored and does not move. Among them, the XOY plane of the world coordinate system of the multi-camera measurement system is in the measurement plane, the Z axis is vertically upward, and the positive direction of the Z axis points to the camera.

[0028] The solution of the marked points on the reconstructed calibration board involves the external parameters of the cameras with a common field of view. Cameras with a common field of view are used to synchronously collect the images of the marked points on the calibration board within the field of view. Given the internal parameters of each camera and the world coordinates of the marked points, the resection method is applied. For the cameras participating in the reconstruction of the marked points, an error equation is established uniformly, and the external parameter values of the cameras with a common field of view are obtained through bundling and adjustment.

[0029] The ID matching determines the cameras for reconstructing the coded marked points and solves the initial values of the external parameters of the cameras participating in the reconstruction of the marked points. It is required that within the measurement area, coded marked points with non-repeating IDs are randomly arranged, and there should be no stacking of the coded marked points. Based on the cameras with existing external parameters, the world coordinates of a part of the coded marked points are first reconstructed; then, in the images of all cameras in the multi-camera measurement system, through the uniqueness of the coded ID, the cameras participating in the reconstruction of the above-mentioned coded marked points except the cameras with existing external parameters are determined; since the number of reconstructed coded marked points is limited, solving the external parameters of the other cameras participating in the reconstruction of the above-mentioned coded marked points (hereinafter referred to as "other cameras") is a special case of the resection method, called single-image resection. In this case, the internal parameters of the camera and the world coordinates of the coded marked points are known, and the corresponding error equation can be expressed as: ; In the formula, represents the residuals of the camera; represents the partial derivative matrix corresponding to the external parameters; L represents the deviation between the actually measured point image coordinates and the initial values of the corresponding theoretical point image coordinates; represents the correction of the world coordinates of the external parameters.

[0030] Since the external parameters of the camera only contain 6 unknowns, only the world coordinates of 3 coded marked points need to be known for the operation. It should be noted that single-image resection is an iterative operation after linearizing the non-linear equation, and it is necessary to give the initial values of the external parameters of the camera before performing the operation to ensure the convergence and solvability of the final equation.

[0031] The method of using single-image resection to solve the initial values of the external parameters of the camera is the pyramid method. Applying the principle that the apex angles between the light rays in the image space and the actual object space of the photographic light beam pyramid are equal, the external parameters corresponding to the camera image are determined. For the specific schematic diagram, see Figure 3 : In the figure, , , are the coordinates of three coded marked points in the world coordinate system respectively, , , They are the image point coordinates of three coded marker points corresponding to a certain camera S. S-xyz is the camera coordinate system, and O-XYZ is the world coordinate system. Among them, the pyramid S- and S- are similar. Since , , and are all unknown, other cameras can be used to capture coded marker points with known world coordinates. By combining the image coordinates corresponding to the marker points captured by other cameras and performing multiple iterative solutions, we can obtain , , and .

[0032] ; ; ; Meanwhile, determine the coordinates , , of the marker points in the S-xyz camera coordinate system , , , which are specifically expressed as: ; ; ; The above-mentioned solution for the initial values of the camera external parameters is to solve the matrix transformation relationship between each camera coordinate system and the world coordinate system. The specific solution process includes: First, solve the coordinates , of the centers of the three fiducial points in the world coordinate system and the camera coordinate system: ; ; Apply the point coordinates , of the centers of the three fiducial points in the world coordinate system and the camera coordinate system, and combine them with the point coordinates , , of the three marker points in the world coordinate system and the point coordinates , , in the camera coordinate system; Solve the rotation matrix from the world coordinate system to the camera coordinate system as: ; Continue to solve the translation matrix from the camera coordinate system to the world coordinate system is: ; wherein, represents the coordinates of the centroid of the three fiducial points in the camera coordinate system; represents the coordinates of the centers of the three fiducial points in the world coordinate system. Thus, the rotation and translation matrices from the world coordinate system to the camera coordinate system of other cameras are obtained 、 , and the transpose of this rotation and translation matrix is the matrix conversion relationship from the camera coordinate system to the world coordinate system of other cameras, and is also the initial value of the external parameters of other cameras 、 . It should be noted that the above method for solving the initial value of the external parameters of other cameras requires at least the world coordinates of 3 coded fiducial points to be known.

[0033] As a preferred technical solution, the method for bundling and optimizing all the external parameter values of the cameras with the existing initial values of the external parameters includes: In view of the foregoing technical solution, the internal parameters of all the cameras in the multi-camera measurement system and the initial values of the external parameters of all the cameras participating in the reconstruction of the coded fiducial points are known, and the error equation of the bundle adjustment is simplified: ; wherein, B and C are the partial derivative matrices corresponding to the external parameters and the world coordinates of the measurement points respectively; During the above optimization process, the centroid coordinates of all the coded fiducial points are kept unchanged as a constraint to ensure that the coordinate system does not shift. The specific method includes: First, 7 coded fiducial points are used as the observations. Since the coded fiducial points are linearly independent of each other, that is, according to the method of conditional least squares adjustment, the following constraint equation E is obtained: ; The physical meaning of this equation is that for all the coded fiducial points including the external parameters of the camera, the centroid and the sum of the vectors from all points to the centroid remain unchanged, that is, all points are anchored at one position and cannot rotate as a whole.

[0034] wherein, d refers to the number of observation points, which is expressed as 7; u refers to the number of unknowns, which is expressed as the external parameters of the camera to be solved and the world coordinates of the coded fiducial points, which is expressed as 12. The first three rows of the equation are the derivatives of the three equations for the unchanged centroid of all the coded fiducial point coordinates with respect to the unknowns to be solved. The middle three rows are the reciprocals of the three equations for the unchanged sum of the vectors from all points to the centroid with respect to the unknowns to be solved. The last row is the constraint for the unchanged ratio of all three-dimensional points. The number of columns of the matrix represents the number of unknowns to be solved.

[0035] The above optimization algorithm is first carried out with 7 coded fiducial points as the observations, and then all the coded fiducial points in the measurement area are used as the observations with this optimization method until all the coded fiducial points have participated in the above optimization algorithm.

[0036] Since a constraint that the barycentric coordinates of the coded marker points remain unchanged is added to the existing error equation, the Lagrange multiplier method is applied to add an augmented matrix of the constraint after the original error equation matrix and add a coefficient matrix to the unknowns to solve the error equation with constraint conditions. The specific expression is as follows: ; represents the coefficients of the error equation for bundle adjustment, which includes two parts, B and C, namely the external parameters and the partial derivative matrix corresponding to the world coordinates of the measurement points; represents the coefficients of the aforementioned constraint equation, that is ; represents the introduced Lagrange factor; represents the external parameters of the camera and the world coordinates of the measurement points obtained by optimized solution, which includes two parts, respectively representing the external parameters of all cameras participating in the reconstruction of the coded marker points and the corrections of the world coordinates of the measurement points (the marker points randomly scattered in the measurement plane).

[0037] The barycenter of the aforementioned coded marker points does not move, which means that during the process of the cameras of the multi-camera measurement system photographing the coded marker points within the measurement area, all the coded marker points remain stationary and do not move.

[0038] As a preferred technical solution, the specific method of cycling the above two algorithm processes to solve the external parameter values of all cameras of the multi-camera measurement system and applying the proportional relationship of the point distances of the scale to correct and output the accurate external parameter values of all cameras includes: First, after the above two algorithm processes are executed for the first time, at least one camera with known external parameters can be added based on the cameras with a common field of view in the multi-camera measurement system; then, use the cameras of the multi-camera measurement system with known external parameters to continue to reconstruct other coded marker points within the measurement area starting from the already reconstructed coded marker points, and solve again to obtain the external parameters of the cameras that may participate in other coded marker points; then, continue the above two algorithm processes until all the coded marker points within the measurement area are reconstructed, that is, the external parameter values of all cameras of the multi-camera measurement system are solved; finally, apply the proportional relationship of the point distances of the scale to correct and output the external parameter values of all cameras of the multi-camera measurement system.

[0039] The above two algorithm processes are cycled until the external parameter values of all cameras in the multi-camera measurement system are solved, solving the problem that the common field of view of the cameras in the multi-camera measurement system is limited and the external parameters of the multi-camera measurement system cannot be solved. Among them, using the pyramid method, only a very small number of coded marker points need to be reconstructed. Starting from the cameras with a common field of view, it continuously spreads outwards to solve the initial values of the external parameters of other cameras. By continuously cycling the algorithm process, all the coded marker points within the measurement area are reconstructed, and finally the external parameter values of all cameras in the multi-camera measurement system are determined.

[0040] The proportional relationship of the point distances of the applied scale is used to correct and output the external parameter values of all cameras in the multi-camera measurement system. First, the measured value of the point distance of the marker points on the scale is known. The multi-camera measurement system with known external parameter values of all cameras is used to photograph the scale and reconstruct the marker points on both sides of the scale to determine the proportional relationship between the measured point distance and the measured point distance; then, with the constraint that the origin of the world coordinate system remains unchanged, according to the above proportional relationship, the measured point distance is corrected to the measured point distance; finally, after correcting the measured point distance, the world coordinate system also undergoes a scaling relationship, and the more accurate external parameter values of all cameras are correspondingly output. 、 As the initial values of the external parameters of the camera system for subsequent initial calibration. Thus, the calibration work of the multi-camera measurement system is completed, and the internal and external parameter values of all cameras in the multi-camera measurement system are determined.

[0041] As a preferred technical solution, the initialization calibration of the multi-camera measurement system includes: First, start the backlight within the measurement area, and the multi-cameras that have completed the full-parameter calibration at the factory synchronously collect and reconstruct the control points on the outer circle of the measurement area, and reconstruct the world coordinates of the control points. ; Then, since the internal parameters of the camera system are fixed values and do not change, an error equation for simplified bundle adjustment is established for all cameras: ; Among them, V represents the residuals of the cameras; B and C are the partial derivative matrices corresponding to the external parameters and the world coordinates of the measurement points respectively; L represents the deviation between the actual measured point image coordinates and the initial values of the corresponding theoretical point image coordinates; represents the correction of the world coordinates of the external parameters; represents the correction of the world coordinates of the measurement points.

[0042] Apply the world coordinates of the control points and the external parameter values of all cameras in the multi-camera measurement system determined in the calibration stage 、 As the initial parameter values participating in the adjustment, the above error equation is bundle-adjusted to obtain the accurate world coordinates of the control points on the outer circle of the measurement area. And initialize the external parameter values of all cameras in the multi-camera measurement system after calibration 、 。

[0043] Finally, combine the accurate world coordinates of the control points after bundle adjustment to solve the mean value of the distances from all control points to the origin of the world coordinate system ; At the same time, record the environmental temperature during initial calibration to complete the initial calibration

[0044] As a preferred technical solution, the dynamic calibration of the multi-camera measurement system includes: First, in the long-term working environment of the multi-camera measurement system, to ensure the measurement accuracy. Before each measurement, it is necessary to perform dynamic calibration on the multi-camera measurement system. First, keep the backlight in the measurement area always on, and the multi-camera measurement system synchronously collects and reconstructs the control points on the outer circle of the area to obtain the world coordinates of the control points ; Then, apply the world coordinates of the reconstructed control points and the accurate world coordinates of the existing control points to determine whether the current measurement meets the measurement requirements; Then, confirm the measurement temperature , and according to the thermal expansion principle, correct the mean value of the distances from all control points measured in the dynamic calibration stage to the origin of the world to the mean value of the theoretical point distances ; Finally, apply the corrected mean value of the point distances , and with the condition that the origin of the world coordinate system remains stationary all the time as a constraint, dynamically calibrate the external parameters of the multi-camera measurement system for this measurement 、 。

[0045] The control points mentioned refer to circular marked points distributed in a circle on the periphery of the measurement area and with passive light emission at the bottom light source, having clear edges, obvious black-and-white contrast, and being easy to identify

[0046] The method of applying the world coordinates of the reconstructed control points and the accurate world coordinates of the existing control points to determine whether the current measurement meets the measurement requirements includes: First, the world coordinates and the world coordinates are in the same world coordinate system. Therefore, based on the least squares theory, using the method that the minimum distance is the homologous point, match and determine and The control points with the same name in it; then, according to the matching result, determine whether the current calibration is valid. Among them, if the number of control points with failed matching or invalidity is no more than 3, then the current calibration is valid; otherwise, the calibration is invalid and no further operation is performed. It should be noted that through the matching of control points, invalid calibrations caused by obstacle occlusion, missing control points, and excessive camera position offset can be filtered out.

[0047] The confirmed measured temperature mentioned above , applying the thermal expansion principle, correct the distance between control points in the diagonal direction of the measurement area as the distance between control points The method specifically includes: First, after measurement and determination, record the measurement environment temperature , combined with the environment temperature during the initial calibration , determine the measurement temperature difference : ; Then, according to the thermal expansion principle, the theoretical mean value of the distances from all control points to the origin of the world coordinate system under long-term light source irradiation can be solved : ; Among them, represents the number of bundle adjustments, represents the th length change caused by the thermal expansion principle during the th real-time calibration,

[0048] Then, according to the world coordinates of the control points obtained by reconstruction , solve the mean value of the distances from all control points measured by the real-time multi-camera measurement system to the world origin ; Finally, solve the mean value of the actually measured point distances and the mean value of the distances between control points in the ideal measurement environment under the influence of thermal expansion to obtain the scale factor, and correct the mean value of the actually measured control point distances to the theoretical mean value of the distances between control points .

[0049] The application of the corrected mean value of the distances between control points , and with the constraint that the origin of the world coordinate system remains stationary all the time, dynamically calibrate the external parameters of the multi-camera measurement system for this measurement , The specific method includes: First, during the calibration process, the internal parameters of all cameras are fixed and remain unchanged. Apply the mean value of the point distances , calibrate the displacement of the calibration control points caused by thermal expansion during measurement; then, with the constraint that the origin of the world coordinate system remains stationary, combine the accurate world coordinates of the control points on the outer circle of the measurement area and the external parameter values of all cameras in the multi-camera measurement system after initialization calibration 、 As the initial parameter values for bundle adjustment, apply the error equation of simplified bundle adjustment to all cameras: ; The external parameters of all cameras in the multi-camera measurement system after real-time calibration are obtained through bundle adjustment 、 . Among them, represents the number of times of bundle adjustment.

[0050] It should be noted that the initialization calibration process is based on the initial external parameters of the multi-camera measurement system determined by system calibration 、 and the world coordinates of the control points reconstructed in the initialization calibration stage as the initial parameter values for bundle adjustment; the dynamic calibration process is based on the external parameters of the multi-camera measurement system determined by initialization calibration 、 and the world coordinates of the global points after initialization calibration as the initial parameter values for bundle adjustment. At the same time, with the constraint that the origin of the world coordinate system remains stationary for bundle adjustment, this way ensures that the external parameter relationship of the multi-camera measurement system is not offset by external factors. While ensuring the measurement accuracy, the external parameters of the multi-camera measurement system after dynamic calibration are more in line with the actual measurement environment.

[0051] Such as Figure 2 shown, the measurement system includes: a box body 9, a camera 1, a tempered glass 3, a backlight source 2, and a controller 7; A plurality of cameras 1 are arranged at intervals around the inner edge of the top of the box body 9; the camera located in the middle area of the box body edge is the central area camera 8; The backlight source 2 is arranged on the bottom end surface of the tempered glass 3, and the coding marker points 4 and the backlight marker points 6 are arranged on the top end surface of the tempered glass 3; The tempered glass 3 is placed on the bottom end surface inside the box body 9, and a marker point scale 5 is arranged on the top end surface of the tempered glass 3; The controller 7 is respectively connected to the camera 1 and the backlight source 2. Among them, all the cameras 1 in the multi-camera measurement system synchronously collect and reconstruct the backlight marking points 6 fixed on the plane of the tempered glass 3. At the same time, the multi-camera measurement system calculates and corrects the offset of the control point distance caused by the thermal expansion effect. Combining with the external parameter values of the cameras in the multi-camera measurement system determined in the system initialization and calibration stage, the real-time calibration of the system is completed. The whole process does not require manual intervention, and the acquisition and calculation time do not exceed 2 s, overcoming the influence of environmental vibration, camera self-heating, and the accuracy of the system itself on the measurement accuracy of the multi-camera measurement system, and truly realizing the real-time self-calibration measurement of the multi-camera measurement system with low cost, high efficiency, and high precision.

[0052] In addition, the present invention also provides an optimized calibration method for the calibration function of the multi-camera measurement system, for the multi-camera measurement system that is undergoing full-parameter calibration at the factory. Through the full-parameter calibration at the factory, a world coordinate system located at the center point of the measurement area is established. Using the corner cube method, the external parameters of the cameras with a common field of view in the center area and the world coordinates of the coded marking points within the common field of view are known, and the initial values of the external parameters of some other cameras are solved by continuously spreading outwards. At the same time, the initial values of the external parameters of all the currently known cameras are bundled and adjusted to obtain the optimized external parameter values of all the cameras participating in the reconstruction of the coded marking points. By analogy, the external parameter values of all the cameras in the optimized multi-camera measurement system are obtained.

[0053] In view of the above optimized calibration method, after the last external parameter solution and bundling adjustment are completed, a scale ruler with a known marking point distance is introduced, and the scaling scale of the world coordinate system is constrained and corrected using the proportional relationship between the measured point distance and the measured point distance, and the more accurate external parameters of all the cameras are output.

[0054] Among them, the inventive point of the present application lies in the full-parameter calibration at the factory: technical optimization is carried out on the basis of the existing calibration algorithm to break its technical limitations and solve the internal and external parameter values of the multi-camera measurement system.

[0055] Internal parameter calibration: Break the requirement for the format of the calibration board in the existing calibration algorithm.

[0056] Apply a dot matrix calibration board printed with coded marking points and non-coded marking points, place multiple poses of the calibration board at various positions in the measurement format, know the world coordinates of all the marking points on the calibration board, and combine the obtained images of the marking points captured by the camera to establish an error equation for all the cameras based on the resection method, and uniformly optimize and adjust to obtain the internal parameter values of each camera.

[0057] External parameter calibration: Break the requirement for the common field of view of the cameras in the existing calibration algorithm.

[0058] Place the calibration board at the center of the measurement area, establish the world coordinate system and solve the external parameter values of some cameras in the central area; then, remove the calibration board and randomly arrange non-repeating coded marker points and a marker point scale with a known point distance. Through coded ID matching, determine all the cameras participating in the reconstruction of each coded marker point; based on the pyramid method, solve the initial values of the external parameters of the other cameras participating in the reconstruction of the coded marker points except for the cameras with known external parameters; for all the aforementioned cameras, uniformly establish an error equation and bundle and optimize the external parameter values of all the cameras; since the coded marker points arranged within the measurement area have filled the measurement area, according to the number of cameras in the multi-camera measurement system, continuously loop through steps b-d until all the external parameter values of the cameras in the multi-camera system are obtained; to ensure that the measurement results conform to the actual physical dimension, introduce a scale with a known point distance. After completing the loop measurement, apply the point distance proportional relationship on the scale to correct and output the accurate external parameter values of all the cameras in the multi-camera measurement system.

[0059] On-site calibration: On-site calibration innovatively introduces the method of reconstructing control points, which solves the problem that the measurement accuracy of the multi-camera measurement system cannot be guaranteed during the actual measurement process.

[0060] Initial calibration: Introduce initial measurement parameters for dynamic calibration to ensure calibration accuracy.

[0061] Use the calibrated multi-camera measurement system to reconstruct the control points within the measurement area for the first time, calibrate to obtain a set of external parameter relationships of the multi-camera measurement system. This set of external parameter relationships will be the initial values of the external parameters for the dynamic calibration of the multi-camera measurement system. At the same time, record the environmental temperature and the average value of the distances from all control points to the origin of the world coordinate system.

[0062] Dynamic calibration: Before each measurement, the multi-camera measurement system reconstructs the control points again to determine whether to perform dynamic calibration.

[0063] Based on the control point-related data obtained from the initial calibration, the multi-camera measurement system synchronously collects and reconstructs the control points within the measurement area again, anchors the world coordinate system, that is, the origin of the world coordinate system remains unchanged, and applies the thermal expansion principle to dynamically calibrate the external parameter relationships of the multi-camera measurement system.

[0064] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A self-calibration method for a multi-camera measurement system, characterized in that The method includes the following steps: Step 1, internal parameter optimization: Arrange a calibration board with coded and non-coded marker points within the measurement field of view. Synchronously acquire images of the calibration board at different poses by multiple cameras. Establish an error equation based on the resection method and perform global optimization to obtain the internal parameter values of each camera; Step 2, initial value solution of external parameters: Take the center point of the calibration board as the origin of the world coordinate system. Obtain the initial external parameters through the reconstruction of the marker points by the cameras within the common field of view. After removing the calibration board, arrange coded marker points. Use the cameras with existing external parameters to reconstruct the coordinates of the marker points, and combine with the corner cube method to match and solve the initial values of the external parameters of other cameras; Step 3, external parameter optimization: Establish an error equation for all cameras, and perform bundle adjustment optimization with the constraint that the centroid coordinates of the marker points remain unchanged to obtain accurate external parameters; Step 4, scale correction: Perform proportional correction on the optimized external parameters according to the scale ratio of the scale points; Step 5, dynamic calibration: Synchronously acquire control points before each measurement, and combine with the environmental temperature data to dynamically calibrate the external parameter relationship based on the principle of thermal expansion.

2. The self-calibration method of the multi-camera measurement system according to claim 1, characterized in that The internal parameter optimization method described in Step 1 includes: Using a dot matrix calibration board of marker points, continuously move the position of the calibration board within the measurement area and place it in different poses at each position. At the same time, all cameras synchronously acquire images of the calibration board. Then, based on the resection method, establish an error equation for all cameras uniformly, and then perform bundling and optimization to obtain the internal parameter values of all cameras; The internal parameter values include: the focal length of the lens , lens distortion ( , , , , , , ), principal point deviation ( , ); Among them, lens distortion refers to the lens distortion caused by the influence of lens processing technology and the assembly position of the lens; ( ) represents the mirror distortion parameter; ( ) represents the decentration distortion parameter; ( ) represents the planar distortion parameter; The principal point deviation represents the deviation of the camera principal point in the camera coordinate system. It represents the deviation of the camera principal point position in the width direction in the camera coordinate system; It represents the deviation of the camera principal point position in the height direction in the camera coordinate system.

3. The self-calibration method of the multi-camera measurement system according to claim 2, characterized in that In Step 2, the solution of the initial value of the external parameters includes: Place the center point of the dot matrix calibration board near the center point within the measurement area, and set this point as the origin of the world coordinate system of the multi-camera measurement system; This point is the coincidence point of the center point of the dot matrix calibration board and the measurement area; In the multi-camera measurement system, there are cameras with a common field of view. Reconstruct the marker points on the calibration board located within the common field of view and that can be photographed, and apply the resection method to first solve the external parameter values of the cameras participating in the marker point reconstruction; Remove the calibration board from the measurement area, and randomly place non-repeating numbered coded marker points. Use the cameras with existing external parameters to photograph and reconstruct the coordinates of the coded marker points that can be photographed; Use the method of ID matching to determine the other cameras that may participate in the reconstruction of the aforementioned marker points except for the cameras with existing external parameters. Based on the corner cube method, combine with the existing marker point coordinates to solve the initial values of the external parameters of the cameras that may participate in the marker point reconstruction; The marker points of the dot matrix calibration board are composed of regularly arranged circular coded points and non-coded points. The spacing between the marker points is fixed and the world coordinates of all marker points on the calibration board are known.

4. The self-calibration method of the multi-camera measurement system according to claim 3, characterized in that, The external parameter values include: (R, T), representing the matrix transformation relationship from the camera coordinate system to the world coordinate system; R represents the rotation matrix relationship from the camera coordinate system to the world coordinate system; T represents the translation matrix relationship from the camera coordinate system to the world coordinate system; Based on the internal parameter values and external parameter values, the error equation is as follows: ; where X1, X2, and X3 are the corrections to the world coordinates of the internal parameters, external parameters, and measurement points respectively; A, B, and C are the partial derivative matrices corresponding to the internal parameters, external parameters, and world coordinates of the measurement points respectively; L represents the deviation between the observed true image point coordinates and the initial value of the theoretical image point coordinates, and the corresponding formula is: ； ； Wherein, V represents the residual of the camera; represents the component of the camera residual on the X-axis of the image coordinate system; represents the component of the camera residual on the Y-axis of the image coordinate system; represents the initial value of the X coordinate of the image point obtained by actual measurement; represents the initial value of the X coordinate of the theoretical point image corresponding to the actually measured image point; represents the initial value of the Y coordinate of the image point obtained by actual measurement; represents the initial value of the Y coordinate of the theoretical point image corresponding to the actually measured image point.

5. The self-calibration method of the multi-camera measurement system according to claim 3, characterized in that The resection method is to establish an error equation for all cameras uniformly, and perform bundling and optimization to obtain the internal parameter values of all cameras; The image point coordinates of the marker points on the calibration plate that are collected by all cameras and cover a certain number of images are taken as observation values, where a certain number means that all cameras are required to collect images of the calibration plate at the same time, and the images of the calibration plate taken by each camera contain at least 40% of the marker point images on the calibration plate; the world coordinates of all the marker points on the calibration plate are known, and since the camera intrinsic and extrinsic parameters corresponding to each image have a total of 16 unknowns, the image point coordinates of 8 marker points in each image can be used to solve the intrinsic and extrinsic parameter values of the camera corresponding to the current image; Among them, after the calibration plate is moved to different positions, the same camera corresponds to multiple images, and the error equations of multiple images are combined to optimize and solve the optimal initial values of the internal and external parameters of all cameras.

6. The self-calibration method of the multi-camera measurement system according to claim 3, characterized in that The ID matching determines the camera for reconstructing the coded mark points, and when solving the initial values of the external parameters of the camera involved in reconstructing the mark points, the coded mark points with non-repeated IDs are randomly arranged within the measurement format; The process of solving the initial values of the camera extrinsic parameters that may be involved in the reconstruction of the marker points includes: based on the camera with existing extrinsic parameter values, firstly reconstructing the world coordinates of a part of the encoded marker points; Continue to match and determine other cameras that may participate in reconstructing the coded marker points in all camera images, except for the cameras with existing external parameter values, through the uniqueness of the coded ID; The single image resection method is used to calculate the initial values of the external parameters of other cameras that may be involved in reconstructing the coded markers. Since the intrinsic parameters of the camera and the world coordinates of the coded markers are known, the corresponding error equation is expressed as: ； Where, V represents the residual of the camera; B represents the partial derivative matrix corresponding to the external parameters; L represents the deviation between the actually measured point image coordinates and the initial values of the corresponding theoretical point image coordinates. represents the correction of the world coordinates of the external parameters.

7. The self-calibration method of the multi-camera measurement system according to claim 6, characterized in that, The method for solving the initial value of the camera extrinsic parameters by single image back intersection is the cone method, that is, the extrinsic parameters corresponding to the camera image are determined by applying the principle that the vertex angles between the image space and the light rays in the actual object space of the photographic beam cone are equal.

8. A multi-camera measurement system, applied to the calibration method described in claims 1-6, characterized in that, The measuring system comprises: a box, a camera, tempered glass, a backlight source and a controller; A plurality of cameras are arranged at intervals around the top inner edge of the box; The backlight source is arranged on the bottom end surface of the tempered glass, and the coding mark point and the backlight mark point are arranged on the top end surface of the tempered glass; The tempered glass is placed on the bottom end surface of the box, and a marking point scale is provided on the top end surface of the tempered glass; The controller is connected to the camera and the backlight source respectively.

Images