Dynamic interference-oriented camera array online collaborative calibration method and system

Abstract

The invention discloses a dynamic interference-oriented camera array online collaborative calibration method and system. The method comprises the following steps: S1, solving camera external parameters of each camera in a camera array; s2, constructing a camera coordinate system linear equation set, and then solving the camera coordinate system linear equation set to obtain coordinates of a three-dimensional calibration point on the calibration object observed by a camera; s3, circulating the step S2 to obtain calibration results of the plurality of cameras; s4, solving the mean value of the actual world coordinates of the three-dimensional calibration points by using the coordinates of the three-dimensional calibration points on the calibration object observed by the plurality of cameras, then solving the Euclidean distance error of the to-be-detected camera according to the mean value of the actual world coordinates of the three-dimensional calibration points, judging whether the camera is abnormal according to the Euclidean distance error of the to-be-detected camera, and if so, judging whether the camera is abnormal or not. And if not, calibration updating is carried out through S1 and S2. According to the method, the stability and the anti-interference capability of calibration parameters in a dynamic environment are remarkably improved, and the high-precision consistency of a multi-camera geometrical relationship is ensured.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of photogrammetry, in particular to a camera array online cooperative calibration method and system for dynamic interference. BACKGROUND

[0002] Camera array systems are widely used in industrial vision detection, three-dimensional reconstruction, augmented reality, robot perception and other fields. In order to ensure system accuracy, the geometric relationship between multiple cameras needs to be obtained through a calibration process. At present, the mainstream camera array calibration method depends on offline calibration, that is, a standard reference object such as a calibration board is used for one-time calibration in a stable environment. However, in actual industrial scenarios, the device will inevitably be affected by various dynamic interference factors during long-term operation, such as platform vibration, structure thermal expansion and contraction caused by temperature changes, image degradation caused by light changes, and small drift of camera position. These dynamic interferences will cause the initial calibration parameters to be invalid, thereby affecting the system measurement accuracy and reliability.

[0003] In order to solve the above problems, some researches try to introduce an online calibration mechanism to cope with environmental changes by continuously collecting images and dynamically adjusting calibration parameters. However, the existing online calibration methods usually rely on specific scene structures or pre-set features, lack the ability to model and compensate for dynamic interference, and have high computational complexity, poor convergence, and insufficient real-time performance in multi-camera cooperative calibration, which makes it difficult to meet the stable operation requirements of high-precision, multi-view systems in dynamic environments. SUMMARY

[0004] The application provides a camera array online cooperative calibration method and system for dynamic interference to solve the technical problems mentioned in the background.

[0005] To achieve the above purpose, the technical scheme of the application is as follows: The application provides a camera array online cooperative calibration method for dynamic interference, comprising the following steps: S1, solving the camera extrinsic parameters of each camera in the camera array, that is, the attitude of each camera in the camera array relative to the calibration board; S2, constructing a camera coordinate system linear equation set according to the camera extrinsic parameters of the camera, and then solving the camera coordinate system linear equation set to obtain the coordinates of the three-dimensional calibration points on the calibration object observed by the camera, that is, the calibration result of the camera; S3, repeating S2 to obtain the coordinates of the three-dimensional calibration points on the calibration object observed by multiple cameras; S4, solving the mean value of the actual world coordinates of the three-dimensional calibration points on the calibration object by using the coordinates of the three-dimensional calibration points observed by the plurality of cameras, then solving the Euclidean distance error of the camera to be detected according to the mean value of the actual world coordinates of the three-dimensional calibration points, and determining whether the camera is abnormal according to the Euclidean distance error of the camera to be detected, if abnormal, updating the calibration through S1 and S2.

[0006] Further, the S1 specifically comprises the following steps: S11, placing the calibration board in the overlapping field of view region of the camera array, collecting F frames of images each time the calibration board is moved or rotated to cover different poses; S12, for each frame of image of each camera in the camera array, using the Harris corner point detection algorithm to extract the pixel coordinates of all corner points of the calibration board to obtain the mutual relationship between the three-dimensional geometric position of the corner points on the calibration board and the corresponding points of the current corner points in the image; S13, solving the final camera extrinsic parameters of all cameras of the camera array relative to the calibration object based on the PnP algorithm.

[0007] Further, the S12 specifically comprises the following steps: S121, first establishing the Harris response function of the zth frame of image, specifically as follows: ; Wherein, represents the response value of the corner point; det represents the determinant of the calculation matrix, is the image gradient covariance matrix of the zth frame of image; represents the trace of the calculation matrix, that is, the sum of the main diagonal elements, is an empirical constant; S122, then comparing the response value of the corner point with the image gradient covariance matrix , screening out the points with response value greater than the image gradient covariance matrix , and using the non-maximum suppression method to delete the repeated points to obtain a plurality of corner points of the zth frame of image; S123, repeating S121 to S122 until a plurality of corresponding corner points are obtained for the F frames of collected images; S124, creating an initial index matrix with the same number of corner points as the calibration board; S125, sorting the detected corner points according to the geometric structure of the calibration board, for each detected corner point, searching for adjacent corner points along the boundary direction, and expanding the initial index matrix according to the distance relationship of the adjacent corner points to obtain an expanded index matrix; S126, then searching for the pixel coordinates corresponding to the corner points on the calibration board on the expanded index matrix.

[0008] Further, the S13 specifically comprises the following steps: S131, first, a projection model of the cameras in the camera array is established, and then the camera extrinsic parameters of the cameras are initialized; wherein R represents a rotation matrix of the camera coordinate system on the camera itself to the calibration object coordinate system; R represents a translation vector of the camera coordinate system on the camera itself to the calibration object coordinate system; S132, according to the projection model of the cameras and the initialized camera extrinsic parameters, the projection error of each corner point on the calibration board is calculated; S133, the Jacobian matrix of the projection error with respect to the initialized camera extrinsic parameters is calculated; S134, the projection error of all corner points on the calibration board is calculated to obtain the projection error vector of all corner points, and then the gradient is calculated according to the projection error vector of all corner points and the Jacobian matrix, wherein the gradient is the first derivative of the projection error with respect to the camera extrinsic parameters; S135, the update step is calculated; S136, then, the camera extrinsic parameters are updated according to the update step and the initialized camera extrinsic parameters to obtain the updated camera extrinsic parameters, and the calculation formula is specifically as follows: S137, the steps S132 to S136 are repeated until the error meets the predetermined requirement, and the pose estimation result of the z-th frame image is obtained, and the pose estimation result is the updated camera extrinsic parameters; S138, the pose estimation results of all F frames of images of the cameras are weightedly averaged to obtain the final camera extrinsic parameters of the cameras; S139, the steps S131 to S138 are repeated until the final camera extrinsic parameters of all cameras in the camera array relative to the calibration object are obtained. ​​​​​​​​​​​​​​​​​​​​​​​​​

[0009] Furthermore, in S131, the camera array contains cameras The projection model is as follows: ; in, Represents the theoretically calculated first... The camera in the first The first observed in the frame The pixel coordinates of each corner point; Indicates camera The known intrinsic parameter matrix, Represents three-dimensional calibration points; Projection error in S132 The formula for calculation is: ; in, Represents Euclidean distance; Indicates the actual calculated number of... The camera in the first The first observed in the frame The theoretical pixel coordinates of each corner point.

[0010] Furthermore, the intrinsic parameter matrix includes focal length, principal point, and distortion parameters.

[0011] Furthermore, the formula for calculating the Jacobian matrix in S133 is: ; in, The first Jacobian matrix represents the first Jacobian matrix. OK; This indicates the calculation of partial derivatives; gradient in S134 The formula for calculation is: ; Where T represents the transpose of the matrix; Update step size in S135 The formula for calculation is: ; in, It is a regularization parameter used to control the update step size. Size, It is the identity matrix. , It is a Hessian matrix.

[0012] Furthermore, step S4 specifically includes the following steps: S21, by camera The projection model of the camera Camera coordinates The details are as follows: (1) in, , , For three-dimensional calibration points The three-dimensional coordinates; at the same time, since the calibration plate is assumed to be placed flat on the ground, And rotation matrix Translation vector The mathematical expression is: (2) (3) In the formula, , , , , , , , , Representing rotation matrices respectively The parameters for each row and column; , , They represent translation vectors respectively. The translation amounts along the x, y, and z axes; S22, Camera The camera coordinates are normalized to obtain the normalized camera coordinates, specifically: (4) in, , , These are the normalized camera coordinates; S23. Substituting formula (4) into formulas (1) to (3), we obtain the following relationship: (5) (6) (7) S24. Substituting formula (7) into formulas (5) and (6), we obtain the following relationship: (8) (9) S25. Construct the linear equation system of the camera coordinate system using formulas (8) and (9), as follows: (10) S26. Transform the linear equations of the camera coordinate system into the following relation: (11) in, and These represent two different intermediate parameters, and , ; S27, intermediate parameters By decomposing the equations, we obtain the following relation: (12) in, and It is an orthogonal matrix. It is a singular value diagonal matrix; S28. Calculate intermediate parameters using formulas (12) and (13). pseudo-reversal The details are as follows: (13) in, right The non-zero singular values ​​are taken in reverse order, and the zero element remains zero during the reciprocal process; S29, Through pseudo-inverse The following relationship is obtained by solving: (14) Then, the coordinates of the three-dimensional calibration point on the calibration object observed by the current camera are obtained by using formula (14).

[0013] Furthermore, step S4 specifically includes the following steps: S41. For the same 3D calibration point, select the observation results from at least three cameras. The observation results are the coordinates of the 3D calibration point on the calibration object observed by the current camera. After reversing the 3D coordinates of each camera, take the average to obtain the mean value of the actual world coordinates of the 3D calibration point. ; S42. In the camera array, select the corresponding 3D calibration point in the camera to be tested. Using the mapping of this 3D calibration point and the mean of the actual world coordinates of the 3D calibration point, calculate the Euclidean distance error between the two. The specific calculation formula is as follows: ; S43. Set error threshold If the Euclidean distance error of the camera to be tested Error threshold If yes, it is determined that the camera to be detected is an abnormal camera, the positioning data output of the camera to be detected is suspended, and calibration updating is performed through S1 and S2, otherwise, it is directly determined that the calibration result of the camera to be detected is correct.

[0014] In another aspect, the application further provides a camera array online cooperative calibration system for dynamic interference, comprising a camera array, and the camera array is calibrated by using the camera array online cooperative calibration method.

[0015] The application has the following beneficial effects: 1. The application can effectively suppress noise interference caused by dynamic interference such as vibration and illumination change in single-frame calibration by using multi-frame synchronous observation (i.e., multiple cameras simultaneously shooting multiple images) and combining multi-view (i.e., multiple views of the same calibration board shot by multiple cameras) redundant data fusion of feature points of the calibration board. 2. The application can dynamically remove observation data of an abnormal camera and trigger local recalibration by real-time detection of the Euclidean distance error, thereby significantly improving the stability and anti-interference ability of the calibration parameters in a dynamic environment and ensuring high-precision consistency of the geometric relationship of multiple cameras.

[0016] 3. The application can convert three-dimensional coordinate solving into efficient linear equation (i.e., camera coordinate system linear equation set) solving by establishing a linear algebra system based on normalized image coordinates (i.e., steps S21 to S26), and can quickly detect the camera in combination with the error threshold, thereby significantly reducing the calculation complexity of online calibration. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The application provides a flowchart of a camera array online cooperative calibration method for dynamic interference. DETAILED DESCRIPTION

[0018] In order to facilitate the understanding of the application, the application will be described more fully below with reference to the accompanying drawings. The preferred embodiments of the application are shown in the drawings. However, the application can be implemented in many other different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the application more thorough and comprehensive.

[0019] Referring to Figure 1 The application provides a camera array online cooperative calibration method for dynamic interference, comprising the following steps: S1, solving camera extrinsics of each camera in the camera array, the camera extrinsics being the attitude of each camera in the camera array relative to the calibration board; the camera array comprises stationary cameras; S2. Construct a system of linear equations for the camera coordinate system based on the camera's extrinsic parameters, and then solve the system of linear equations to obtain the coordinates of a three-dimensional calibration point on the calibration object observed by the camera, which is the calibration result of the camera; in this embodiment, the calibration object is a checkerboard pattern. S3, repeat S2, to obtain the coordinates of the three-dimensional calibration points on the calibration object observed by multiple cameras; S4. Calculate the average real-world coordinates of the three-dimensional calibration points on the calibration object using the coordinates observed by multiple cameras. Then, calculate the Euclidean distance error of the camera under test based on the average real-world coordinates of the three-dimensional calibration points. Determine whether the camera is abnormal based on the Euclidean distance error of the camera under test. If abnormal, update the calibration through S1 and S2.

[0020] This invention effectively suppresses noise interference caused by dynamic disturbances such as vibration and illumination changes in single-frame calibration by combining multi-frame synchronous observation (i.e., multiple images captured by multiple cameras simultaneously) with redundant data fusion of multi-view (i.e., multiple views of the same calibration board captured by multiple cameras) of feature points of the calibration board.

[0021] In some embodiments, S1 specifically includes the following steps: S11. Place the calibration board within the overlapping field of view of the camera array. Each time the calibration board is moved or rotated, acquire one frame of image. Acquire a total of F frames of images to cover different poses. S12. For each frame of image from each camera in the camera array, the Harris corner detection algorithm is used to extract the pixel coordinates of all corner points on the calibration board, so as to obtain the relationship between the three-dimensional geometric position of the corner point on the calibration board and the corresponding point of the current corner point in the image; the pixel coordinates are as follows: ; in, For the first The camera at the The first frame observed in the image The position of each corner point N represents the pixel coordinates; N indicates the first pixel coordinate. The total number of corner points in a frame image; S13. Solve the final camera extrinsic parameters of all cameras in the camera array relative to the calibration object based on the PnP (Perspective-n-Point) algorithm.

[0022] In some embodiments, S12 specifically includes the following steps: S121. First, establish the Harris response function for the z-th frame image, as follows: ; in, Represents the corner response value; det represents the determinant of the calculated matrix. Let z be the image gradient covariance matrix of the z-th frame; This represents calculating the trace of a matrix, which is the sum of the elements on the main diagonal. This is an empirical constant, typically ranging from 0.04 to 0.06; S122, then the diagonal response value With the image gradient covariance matrix Compare and filter out responses with values ​​greater than the image gradient covariance matrix. The points are identified, and duplicate points are removed using the non-maximum suppression method to obtain multiple corner points of the z-th frame image; S123, repeat S121 to S122 until multiple corresponding corner points are obtained for all F frames of images collected; S124. Create an initial index matrix with the same number of corner points as the calibration board; S125. Sort the detected corner points according to the geometry of the calibration plate. For example, the corner points of a checkerboard can be arranged in order from left to right and from top to bottom. For each detected corner point, search for neighboring corner points along the boundary directions (up, down, left, right), and expand the initial index matrix according to the distance relationship of neighboring corner points to obtain the expanded index matrix; S126. Then, find the pixel coordinates corresponding to the corner points on the calibration board on the amplified index matrix.

[0023] In some embodiments, S13 specifically includes the following steps: S131. First, establish the cameras in the camera array. The projection model is then initialized, and the camera is initialized. Camera external parameters ,in Indicates camera The rotation matrix from the camera coordinate system to the calibration object coordinate system; Indicates camera The translation vector from the camera coordinate system to the calibration object coordinate system; S132, Based on the camera Projection model and initialized camera extrinsic parameters Calculate the projection error of each corner point on the calibration plate. ; S133, Calculate projection error Regarding the camera extrinsic parameters after initialization Jacobian matrix ; S134. Calculate the projection error of all corner points on the calibration board to obtain the projection error vector of all corner points. Then, based on the projection error vector of all corner points And Jacobi matrix Calculate gradient ,gradient It is projection error Regarding camera external parameters The first derivative; S135, Calculate the update step size ; S136, then based on the update step size and the camera extrinsic parameters after initialization Update camera extrinsic parameters to obtain updated camera extrinsic parameters. The specific calculation formula is as follows: ; S137, repeat S132 to S136 until error occurs. If the predetermined requirements are met, the pose estimation result of the z-th frame image is obtained, and the pose estimation result is the updated camera extrinsic parameters; S138, Camera The pose estimation results of all F frames are weighted and averaged to obtain the camera... Final camera extrinsic parameters; S139, repeat S131 to S138 until the final camera extrinsic parameters of all cameras in the camera array relative to the calibration object are obtained.

[0024] In some embodiments, the cameras in the camera array in S131 The projection model is as follows: ; in, Represents the theoretically calculated first... The camera at the The first observed in the frame The pixel coordinates of each corner point; Indicates camera The known intrinsic parameter matrix, Represents three-dimensional calibration points; Projection error in S132 The formula for calculation is: ; in, Represents Euclidean distance; Indicates the actual calculated number of... The camera at the The first observed in the frame Theoretical pixel coordinates of a corner point.

[0025] In some embodiments, the intrinsic matrix comprises focal length, principal point and distortion parameters.

[0026] In some embodiments, the Jacobian matrix in S133 is calculated as follows: ; wherein, represents the i-th row of the Jacobian matrix; ; represents the partial derivative calculation; The gradient in S134 is calculated as follows: ; wherein, T represents the transpose of a matrix; The update step in S135 is calculated as follows: ; wherein, is a regularization parameter for controlling the size of the update step , is an identity matrix, , is a Hessian matrix.

[0027] In some embodiments, the S4 specifically comprises the following steps: S21, obtaining the camera coordinates of the camera by the projection model of the camera , specifically as follows: (1) wherein, , , is the three-dimensional coordinate of the three-dimensional calibration point ; at the same time, since the calibration board is by default placed on the ground, therefore ; and the mathematical expressions of the rotation matrix and the translation vector are as follows: (2) (3) wherein, , , , , , , , ​​​​, respectively represent the parameters of each row and each column in the rotation matrix ; , , respectively represent the translation amounts of the x, y, z three axes in the translation vector ; S22, normalize the camera coordinates of the camera to obtain normalized camera coordinates, specifically as follows: (4) wherein, , , are the normalized camera coordinates; S23, substitute formula (4) into formula (1) to formula (3) to obtain the following relationship formulas: (5) (6) (7) S24, substitute formula (7) into formula (5) and formula (6) to obtain the following relationship formulas: (8) (9) S25, construct the camera coordinate system linear equation group by using formula (8) and formula (9), specifically as follows: (10) S26, convert the camera coordinate system linear equation group into the following relationship formula: (11) wherein, and respectively represent two different intermediate parameters, and , ; S27, decompose the intermediate parameter to obtain the following relationship formula: (12) wherein, and are orthogonal matrices, is a singular value diagonal matrix; S28, calculate the pseudo-inverse of the intermediate parameter by using formula (12) and formula (13), specifically as follows: (13) in, right The non-zero singular values ​​are taken in reverse order, and the zero element remains zero during the reciprocal process; S29, Through pseudo-inversion The following relationship is obtained by solving: (14) Then, the coordinates of the three-dimensional calibration point on the calibration object observed by the current camera are obtained by using formula (14).

[0028] In some embodiments, S4 specifically includes the following steps: S41. For the same 3D calibration point, select the observation results from at least three cameras. The observation results are the coordinates of the 3D calibration point on the calibration object observed by the current camera. After reversing the 3D coordinates of each camera, take the average to obtain the mean value of the actual world coordinates of the 3D calibration point. ; S42. In the camera array, select the corresponding 3D calibration point in the camera to be tested. Using the mapping of this 3D calibration point and the mean of the actual world coordinates of the 3D calibration point, calculate the Euclidean distance error between the two. The specific calculation formula is as follows: ; S43. Set error threshold If the Euclidean distance error of the camera to be tested Error threshold If the test camera is found to be abnormal, the output of its positioning data will be paused, and calibration will be updated via S1 and S2. Otherwise, the calibration result of the test camera will be considered correct.

[0029] This invention can dynamically eliminate observation data from abnormal cameras by real-time detection of Euclidean distance error and trigger local recalibration, which significantly improves the stability and anti-interference ability of calibration parameters in dynamic environments and ensures high-precision consistency of geometric relationships among multiple cameras.

[0030] This invention establishes a linear algebraic system based on normalized image coordinates (i.e., steps S21 to S26), transforming the solution of three-dimensional coordinates into an efficient solution of linear equations (i.e., a system of linear equations for the camera coordinate system). Combined with an error threshold, this invention enables rapid camera detection and significantly reduces the computational complexity of online calibration.

[0031] In another aspect, the present invention provides an online collaborative calibration system for camera arrays accommodating dynamic interference, comprising a camera array, wherein the camera array is calibrated using an online collaborative calibration method for camera arrays.

[0032] The above merely describes specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Furthermore, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the realization of the ordinary skilled person in the art, when the combination of the technical solutions appears contradictory or unachievable, it should be considered that the combination of the technical solutions does not exist, and is not within the protection scope required by the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for online collaborative calibration of camera arrays accommodating dynamic interference, characterized in that, Includes the following steps: S1. Solve for the camera extrinsic parameters of each camera in the camera array. The camera extrinsic parameters are the attitudes of each camera in the camera array relative to the calibration board. S2. Construct a system of linear equations for the camera coordinate system based on the camera's extrinsic parameters, and then solve the system of linear equations for the camera coordinate system to obtain the coordinates of a three-dimensional calibration point on the calibration object observed by the camera, which is the calibration result of the camera. S3, repeat S2, to obtain the coordinates of the three-dimensional calibration points on the calibration object observed by multiple cameras; S4. Calculate the average real-world coordinates of the three-dimensional calibration points on the calibration object using the coordinates observed by multiple cameras. Then, calculate the Euclidean distance error of the camera under test based on the average real-world coordinates of the three-dimensional calibration points. Determine whether the camera is abnormal based on the Euclidean distance error of the camera under test. If abnormal, update the calibration through S1 and S2.

2. The online collaborative calibration method for camera arrays accommodating dynamic interference as described in claim 1, characterized in that, S1 specifically includes the following steps: S11. Place the calibration board within the overlapping field of view of the camera array. Each time the calibration board is moved or rotated, acquire one frame of image. Acquire a total of F frames of images to cover different poses. S12. For each frame of image from each camera in the camera array, the Harris corner detection algorithm is used to extract the pixel coordinates of all corners of the calibration board, so as to obtain the relationship between the three-dimensional geometric position of the corners on the calibration board and the corresponding point of the current corner in the image. S13. Solve the final camera extrinsic parameters of all cameras in the camera array relative to the calibration object based on the PnP algorithm.

3. The online collaborative calibration method for camera arrays accommodating dynamic interference as described in claim 2, characterized in that, S12 specifically includes the following steps: S121. First, establish the Harris response function for the z-th frame image, as follows: ； in, Represents the corner response value; det represents the determinant of the calculated matrix. Let z be the image gradient covariance matrix of the z-th frame; This represents calculating the trace of a matrix, which is the sum of the elements on the main diagonal. These are empirical constants; S122, then the diagonal response value With the image gradient covariance matrix Compare and filter out responses with values ​​greater than the image gradient covariance matrix. The points are identified, and duplicate points are removed using the non-maximum suppression method to obtain multiple corner points of the z-th frame image; S123, repeat S121 to S122 until multiple corresponding corner points are obtained for all F frames of images collected; S124. Create an initial index matrix with the same number of corner points as the calibration board; S125. Sort the detected corner points according to the geometry of the calibration plate. For each detected corner point, search for neighboring corner points along the boundary direction and expand the initial index matrix according to the distance relationship of neighboring corner points to obtain the expanded index matrix. S126. Then, find the pixel coordinates corresponding to the corner points on the calibration board on the amplified index matrix.

4. The online collaborative calibration method for camera arrays accommodating dynamic interference as described in claim 3, characterized in that, S13 specifically includes the following steps: S131. First, establish the cameras in the camera array. The projection model is then initialized, and the camera is initialized. Camera external parameters ,in Indicates camera The rotation matrix from the camera coordinate system to the calibration object coordinate system; Indicates camera The translation vector from the camera coordinate system to the calibration object coordinate system; S132, Based on the camera Projection model and initialized camera extrinsic parameters Calculate the projection error of each corner point on the calibration plate. ; S133, Calculate projection error Regarding the camera extrinsic parameters after initialization Jacobian matrix ; S134. Calculate the projection error of all corner points on the calibration board to obtain the projection error vector of all corner points. Then, based on the projection error vector of all corner points And Jacobi matrix Calculate gradient ,gradient It is projection error Regarding camera external parameters The first derivative; S135, Calculate the update step size ; S136, then based on the update step size and the camera extrinsic parameters after initialization Update camera extrinsic parameters to obtain updated camera extrinsic parameters. The specific calculation formula is as follows: ； S137, repeat S132 to S136 until error occurs. If the predetermined requirements are met, the pose estimation result of the z-th frame image is obtained, and the pose estimation result is the updated camera extrinsic parameters; S138, Camera The pose estimation results of all F frames are weighted and averaged to obtain the camera... Final camera extrinsic parameters; S139, repeat S131 to S138 until the final camera extrinsic parameters of all cameras in the camera array relative to the calibration object are obtained.

5. The online collaborative calibration method for camera arrays accommodating dynamic interference according to claim 4, characterized in that, The camera array in S131 contains cameras The projection model is as follows: ； in, Represents the theoretically calculated first... The camera in the first The first observed in the frame The pixel coordinates of each corner point; Indicates camera The known intrinsic parameter matrix, Represents three-dimensional calibration points; Projection error in S132 The formula for calculation is: ； in, Represents Euclidean distance; Indicates the actual calculated number of... The camera in the first The first observed in the frame The theoretical pixel coordinates of each corner point.

6. The online collaborative calibration method for camera arrays accommodating dynamic interference according to claim 5, characterized in that, The intrinsic parameter matrix includes focal length, principal point, and distortion parameters.

7. The online collaborative calibration method for camera arrays accommodating dynamic interference as described in claim 5, characterized in that, The formula for calculating the Jacobian matrix in S133 is as follows: ； in, The first Jacobian matrix represents the first Jacobian matrix. OK; This indicates the calculation of partial derivatives; gradient in S134 The formula for calculation is: ； Where T represents the transpose of the matrix; Update step size in S135 The formula for calculation is: ； in, It is a regularization parameter used to control the update step size. Size, It is the identity matrix. , It is a Hessian matrix.

8. The online collaborative calibration method for camera arrays accommodating dynamic interference according to claim 7, characterized in that, S4 specifically includes the following steps: S21, by camera The projection model of the camera Camera coordinates The details are as follows: （1） in, , , For three-dimensional calibration points The three-dimensional coordinates; at the same time, since the calibration plate is assumed to be placed flat on the ground, And rotation matrix Translation vector The mathematical expression is: （2） （3） In the formula, , , , , , , , , Representing rotation matrices respectively The parameters for each row and column; , , They represent translation vectors respectively. The translation amounts along the x, y, and z axes; S22, Camera The camera coordinates are normalized to obtain the normalized camera coordinates, specifically: （4） ； in, , , These are the normalized camera coordinates; S23. Substituting formula (4) into formulas (1) to (3), we obtain the following relationship: （5） （6） （7） S24. Substituting formula (7) into formulas (5) and (6), we obtain the following relationship: （8） （9） S25. Construct the linear equation system of the camera coordinate system using formulas (8) and (9), as follows: （10） S26. Transform the linear equations of the camera coordinate system into the following relation: （11） in, and These represent two different intermediate parameters, and , ; S27, intermediate parameters By decomposing the equations, we obtain the following relation: （12） in, and It is an orthogonal matrix. It is a singular value diagonal matrix; S28. Calculate intermediate parameters using formulas (12) and (13). pseudo-reversal The details are as follows: （13） in, right The non-zero singular values ​​are taken in reverse order, and the zero element remains zero during the reciprocal process; S29, Through pseudo-inversion The following relationship is obtained by solving: （14） Then, the coordinates of the three-dimensional calibration point on the calibration object observed by the current camera are obtained by using formula (14).

9. The online collaborative calibration method for camera arrays accommodating dynamic interference as described in claim 8, characterized in that, S4 specifically includes the following steps: S41. For the same 3D calibration point, select the observation results from at least three cameras. The observation results are the coordinates of the 3D calibration point on the calibration object observed by the current camera. After reversing the 3D coordinates of each camera, take the average to obtain the mean value of the actual world coordinates of the 3D calibration point. ; S42. In the camera array, select the corresponding 3D calibration point in the camera to be tested. Using the mapping of this 3D calibration point and the mean of the actual world coordinates of the 3D calibration point, calculate the Euclidean distance error between the two. The specific calculation formula is as follows: ； S43. Set error threshold If the Euclidean distance error of the camera to be tested Error threshold If the camera under test is found to be abnormal, the output of its positioning data will be paused, and calibration will be updated via S1 and S2. Otherwise, the calibration result of the camera under test will be considered correct.

10. An online collaborative calibration system for camera arrays resistant to dynamic interference, characterized in that, The camera array is calibrated using the online collaborative calibration method for camera arrays as described in any one of claims 1 to 9.

Images