A camera array online cooperative calibration method and system for dynamic interference

Abstract

A camera array online cooperative calibration method and system facing dynamic interference, the method comprising: S1, solving the camera external parameters of each camera in the camera array; S2, constructing a camera coordinate system linear equation group, then solving the camera coordinate system linear equation group to obtain the coordinates of three-dimensional calibration points on a calibration object observed by a camera; S3, circulating S2 to obtain the calibration results of multiple cameras; S4, solving the actual world coordinate mean of three-dimensional calibration points by using the coordinates of three-dimensional calibration points on a calibration object observed by multiple cameras, then solving the Euclidean distance error of the to-be-detected camera according to the actual world coordinate mean of three-dimensional calibration points, and determining whether the camera is abnormal according to the Euclidean distance error of the to-be-detected camera, if abnormal, updating the calibration through S1 and S2. The application significantly improves the stability and anti-interference ability of the calibration parameters in a dynamic environment, and ensures the high-precision consistency of the geometric relationship of multiple cameras.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of photogrammetry, in particular to a camera array online cooperative calibration method and system for dynamic interference. BACKGROUND

[0002] Camera array systems are widely used in industrial vision detection, three-dimensional reconstruction, augmented reality, robot perception and other fields. In order to ensure system accuracy, the geometric relationship between multiple cameras needs to be obtained through the calibration process. The current mainstream camera array calibration method relies on offline calibration, that is, using a calibration board and other standard reference objects for one-time calibration in a stable environment. However, in actual industrial scenarios, the device will inevitably be affected by various dynamic interference factors during long-term operation, such as platform vibration, structural thermal expansion and contraction caused by temperature changes, image degradation caused by light changes, and small drift of camera position. These dynamic interferences will cause the initial calibration parameters to be invalid, thereby affecting the system measurement accuracy and reliability.

[0003] In order to solve the above problems, some researches have tried to introduce online calibration mechanism, which continuously collects images and dynamically adjusts calibration parameters to cope with environmental changes. However, the existing online calibration methods usually rely on specific scene structures or pre-set features, lack the ability to model and compensate for dynamic interference, and have high computational complexity, poor convergence, and insufficient real-time performance in multi-camera cooperative calibration, which makes it difficult to meet the stable operation requirements of high-precision, multi-view systems in dynamic environments. SUMMARY

[0004] The present application provides a camera array online cooperative calibration method and system for dynamic interference to solve the technical problems mentioned in the background art.

[0005] To achieve the above purpose, the technical scheme of the present application is as follows:

[0006] The present application provides a camera array online cooperative calibration method for dynamic interference, comprising the following steps:

[0007] S1, solving the camera extrinsic parameters of each camera in the camera array, the camera extrinsic parameters being the attitude of each camera in the camera array relative to the calibration board;

[0008] S2, constructing a camera coordinate system linear equation set according to the camera extrinsic parameters of the camera, and then solving the camera coordinate system linear equation set to obtain the coordinates of the three-dimensional calibration points on the calibration object observed by the camera, i.e. the calibration result of the camera;

[0009] S3, repeating S2 to obtain the coordinates of the three-dimensional calibration points on the calibration object observed by multiple cameras;

[0010] S4, solving the mean value of the three-dimensional calibration point actual world coordinates by using the coordinates of the three-dimensional calibration points on the calibration object observed by the plurality of cameras, then solving the Euclidean distance error of the to-be-detected camera according to the mean value of the three-dimensional calibration point actual world coordinates, and determining whether the camera is abnormal according to the Euclidean distance error of the to-be-detected camera, if abnormal, updating the calibration through S1 and S2.

[0011] Further, the S1 specifically comprises the following steps:

[0012] S11, placing the calibration board in the overlapping field of view region of the camera array, collecting F frames of images each time the calibration board is moved or rotated to cover different poses;

[0013] S12, for each frame of image of each camera in the camera array, using the Harris corner point detection algorithm to extract the pixel coordinates of all corner points of the calibration board to obtain the mutual relationship between the three-dimensional geometric position of the corner points on the calibration board and the corresponding points of the current corner points in the image;

[0014] S13, solving the final camera extrinsic parameters of all cameras of the camera array relative to the calibration object based on the PnP algorithm.

[0015] Further, the S12 specifically comprises the following steps:

[0016] S121, first establishing the Harris response function of the zth frame of image, specifically as follows:

[0017] ;

[0018] Wherein, represents the corner response value; det represents the determinant of the calculation matrix, is the image gradient covariance matrix of the zth frame of image; represents the trace of the calculation matrix, that is, the sum of the main diagonal elements, is an empirical constant;

[0019] S122, then comparing the corner response value with the image gradient covariance matrix , screening out the points with response value greater than the image gradient covariance matrix , and using the non-maximum suppression method to delete the repeated points to obtain a plurality of corner points of the zth frame of image;

[0020] S123, repeating S121 to S122 until a plurality of corresponding corner points are obtained for the F frames of collected images;

[0021] S124, creating an initial index matrix with the same number of corner points of the calibration board;

[0022] S125, according to the geometric structure of the calibration board, the detected corner points are sorted, for each detected corner point, adjacent corner point searching is performed along the boundary direction, and the initial index matrix is expanded according to the distance relationship of the adjacent corner points to obtain an expanded index matrix;

[0023] S126, then find the pixel coordinates corresponding to the corner points on the calibration board on the expanded index matrix.

[0024] Further, the S13 specifically comprises the following steps:

[0025] S131, first, a projection model of a camera in the camera array is established, and then a camera extrinsic parameter of the camera is initialized, wherein represents a rotation matrix of a camera coordinate system on the camera itself to a calibration object coordinate system; represents a translation vector of the camera coordinate system on the camera itself to the calibration object coordinate system;

[0026] S132, according to the projection model of the camera and the initialized camera extrinsic parameter , a projection error of each corner point on the calibration board is calculated;

[0027] S133, a Jacobian matrix of the projection error with respect to the initialized camera extrinsic parameter is calculated;

[0028] S134, the projection errors of all the corner points on the calibration board are calculated to obtain a projection error vector of all the corner points, and then a gradient is calculated according to the projection error vector of all the corner points and the Jacobian matrix , the gradient is a first derivative of the projection error with respect to the camera extrinsic parameter ;

[0029] S135, an update step length is calculated;

[0030] S136, then, the update step length and the initialized camera extrinsic parameter are used to update the camera extrinsic parameter to obtain an updated camera extrinsic parameter , and the calculation formula is specifically:

[0031] ;

[0032] S137, repeat S132 to S136 until error occurs. If the predetermined requirements are met, the pose estimation result of the z-th frame image is obtained, and the pose estimation result is the updated camera extrinsic parameters;

[0033] S138, Camera The pose estimation results of all F frames are weighted and averaged to obtain the camera... Final camera extrinsic parameters;

[0034] S139, repeat S131 to S138 until the final camera extrinsic parameters of all cameras in the camera array relative to the calibration object are obtained.

[0035] Furthermore, in S131, the camera array contains cameras The projection model is as follows:

[0036] ;

[0037] in, Represents the theoretically calculated first... The camera at the The first observed in the frame The pixel coordinates of each corner point; Indicates camera The known intrinsic parameter matrix, Represents three-dimensional calibration points;

[0038] Projection error in S132 The formula for calculation is:

[0039] ;

[0040] in, Represents Euclidean distance; Indicates the actual calculated number of... The camera in the first The first observed in the frame The theoretical pixel coordinates of each corner point.

[0041] Furthermore, the intrinsic parameter matrix includes focal length, principal point, and distortion parameters.

[0042] Furthermore, the formula for calculating the Jacobian matrix in S133 is:

[0043] ;

[0044] in, The first Jacobian matrix represents the first Jacobian matrix. OK; This indicates the calculation of partial derivatives;

[0045] gradient in S134 The formula for calculation is:

[0046] ;

[0047] Where T represents the transpose of the matrix;

[0048] Update step size in S135 The formula for calculation is:

[0049] ;

[0050] in, It is a regularization parameter used to control the update step size. Size, It is the identity matrix. , It is a Hessian matrix.

[0051] Furthermore, step S4 specifically includes the following steps:

[0052] S21, by camera The projection model of the camera Camera coordinates The details are as follows:

[0053] (1)

[0054] in, , , For three-dimensional calibration points The three-dimensional coordinates; at the same time, since the calibration plate is assumed to be placed flat on the ground, And rotation matrix Translation vector The mathematical expression is:

[0055] (2)

[0056] (3)

[0057] In the formula, , , , , , , , , Representing rotation matrices respectively The parameters for each row and column; , , respectively represent translation vectors translation amounts of the three axes of x, y, and z in the middle;

[0058] S22, normalize the camera coordinates of the camera to obtain normalized camera coordinates, specifically:

[0059] (4)

[0060] wherein, , , the normalized camera coordinates;

[0061] S23, substitute formula (4) into formula (1) to formula (3) to obtain the following relationship:

[0062] (5)

[0063] (6)

[0064] (7)

[0065] S24, substitute formula (7) into formula (5) and formula (6) to obtain the following relationship:

[0066] (8)

[0067] (9)

[0068] S25, construct the camera coordinate system linear equation group by using formula (8) and formula (9), specifically as follows:

[0069] (10)

[0070] S26, convert the camera coordinate system linear equation group into the following relationship:

[0071] (11)

[0072] wherein, and respectively represent two different intermediate parameters, and , ;

[0073] S27, decompose the intermediate parameter to obtain the following relationship:

[0074] (12)

[0075] wherein, and are orthogonal matrices, is a singular value diagonal matrix;

[0076] S28, calculate the intermediate parameter using formula (12) and formula (13) , specifically as follows:

[0077] (13)

[0078] wherein, take the reciprocal of the non-zero singular values in , and the zero elements remain zero during the reciprocal process;

[0079] S29, solve the following relationship formula by pseudo-inverse :

[0080] (14)

[0081] Then, the coordinates of the three-dimensional calibration point on the calibration object observed by the current camera are solved using formula (14).

[0082] Further, the S4 specifically comprises the following steps:

[0083] S41, for the same three-dimensional calibration point, select the observation results of at least three cameras, that is, the coordinates of the three-dimensional calibration point on the calibration object observed by the current camera; respectively, the three-dimensional coordinates are back calculated and averaged to obtain the actual world coordinate mean value of the three-dimensional calibration point .

[0084] S42, in the camera array, select the corresponding three-dimensional calibration point in the to-be-detected camera, and calculate the Euclidean distance error between the three-dimensional calibration point mapping and the actual world coordinate mean value of the three-dimensional calibration point, and the calculation formula is specifically as follows:

[0085] .

[0086] S43, set an error threshold , if the Euclidean distance error of the to-be-detected camera is greater than the error threshold , it is determined that the to-be-detected camera is an abnormal camera, the positioning data output of the to-be-detected camera is suspended, and the calibration update is performed through S1 and S2, otherwise, it is directly determined that the calibration result of the to-be-detected camera is correct.

[0087] Another aspect of the present application also provides a camera array online collaborative calibration system facing dynamic interference, comprising a camera array, which is calibrated by the camera array online collaborative calibration method.

[0088] The present application has the following beneficial effects:

[0089] 1、The present application can effectively suppress the noise interference caused by vibration, illumination change and other dynamic interference in single-frame calibration by multi-frame synchronous observation (i.e., multiple cameras simultaneously shooting multiple images), combined with the multi-view (i.e., multiple views of the same calibration board shot by multiple cameras) redundant data fusion of the feature points of the calibration board.

[0090] 2、The present application can dynamically exclude the observation data of abnormal cameras through real-time detection of the Euclidean distance error, and trigger local re-calibration, significantly improving the stability and anti-interference ability of the calibration parameters in dynamic environment, and ensuring the high-precision consistency of the geometric relationship of multiple cameras.

[0091] 3、The present application can convert the three-dimensional coordinate solving into efficient linear equation (i.e., camera coordinate system linear equation group) solving by establishing a linear algebra system based on normalized image coordinates (i.e., steps S21 to S26), and combined with the error threshold, the camera can be quickly detected, and the calculation complexity of online calibration is significantly reduced. BRIEF DESCRIPTION OF DRAWINGS

[0092] Figure 1 The flowchart of the camera array online collaborative calibration method facing dynamic interference in the present application. DETAILED DESCRIPTION

[0093] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the related drawings. The preferred embodiments of the present application are shown in the drawings. However, the present application can be realized in many other different forms, and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0094] Referring to Figure 1 The present application embodiment provides a camera array online collaborative calibration method facing dynamic interference, comprising the following steps:

[0095] S1, solving the camera extrinsic parameters of each camera in the camera array, the camera extrinsic parameters being the attitude of each camera in the camera array relative to the calibration board; the camera array comprises stationary cameras;

[0096] S2. Construct a system of linear equations for the camera coordinate system based on the camera's extrinsic parameters, and then solve the system of linear equations to obtain the coordinates of a three-dimensional calibration point on the calibration object observed by the camera, which is the calibration result of the camera; in this embodiment, the calibration object is a checkerboard pattern.

[0097] S3, repeat S2, to obtain the coordinates of the three-dimensional calibration points on the calibration object observed by multiple cameras;

[0098] S4. Calculate the average real-world coordinates of the three-dimensional calibration points on the calibration object using the coordinates observed by multiple cameras. Then, calculate the Euclidean distance error of the camera under test based on the average real-world coordinates of the three-dimensional calibration points. Determine whether the camera is abnormal based on the Euclidean distance error of the camera under test. If abnormal, update the calibration through S1 and S2.

[0099] This invention effectively suppresses noise interference caused by dynamic disturbances such as vibration and illumination changes in single-frame calibration by combining multi-frame synchronous observation (i.e., multiple images captured by multiple cameras simultaneously) with redundant data fusion of multi-view (i.e., multiple views of the same calibration board captured by multiple cameras) of feature points of the calibration board.

[0100] In some embodiments, S1 specifically includes the following steps:

[0101] S11. Place the calibration board within the overlapping field of view of the camera array. Each time the calibration board is moved or rotated, acquire one frame of image. Acquire a total of F frames of images to cover different poses.

[0102] S12. For each frame of image from each camera in the camera array, the Harris corner detection algorithm is used to extract the pixel coordinates of all corner points on the calibration board, so as to obtain the relationship between the three-dimensional geometric position of the corner point on the calibration board and the corresponding point of the current corner point in the image; the pixel coordinates are as follows:

[0103] ;

[0104] in, For the first The camera in the first The first frame observed in the image The position of each corner point N represents the pixel coordinates; N indicates the first pixel coordinate. The total number of corner points in a frame image;

[0105] S13. Solve the final camera extrinsic parameters of all cameras in the camera array relative to the calibration object based on the PnP (Perspective-n-Point) algorithm.

[0106] In some embodiments, S12 specifically includes the following steps:

[0107] S121, first, the Harris response function of the zth frame image is established, specifically as follows:

[0108] ;

[0109] wherein, represents the corner response value; det represents the determinant of the calculation matrix, is the image gradient covariance matrix of the zth frame image; represents the trace of the calculation matrix, that is, the sum of the main diagonal elements, is an empirical constant, usually 0.04 to 0.06;

[0110] S122, then the corner response value is compared with the image gradient covariance matrix , the points with response value greater than the image gradient covariance matrix are screened out, and the repeated points are deleted using the non-maximum suppression method, to obtain a plurality of corner points of the zth frame image;

[0111] S123, the S121 to S122 are cycled until the F frame images collected are all obtained a plurality of corresponding corner points;

[0112] S124, an initial index matrix with the same number of corner points as the calibration board is created;

[0113] S125, the detected corner points are sorted according to the geometric structure of the calibration board, for example, the corner points of the chessboard can be arranged in the order from left to right and from top to bottom;

[0114] For each detected corner point, the adjacent corner point search is performed along the boundary direction (up, down, left, right), and the initial index matrix is expanded according to the distance relationship of the adjacent corner points to obtain an expanded index matrix;

[0115] S126, then the pixel coordinates corresponding to the corner points on the calibration board are found on the expanded index matrix.

[0116] In some embodiments, the S13 specifically comprises the following steps:

[0117] S131, first, the projection model of the camera in the camera array is established, and then the camera extrinsic parameter of the camera is initialized, wherein represents the rotation matrix of the camera coordinate system on the camera itself to the calibration object coordinate system; represents the translation matrix of the camera coordinate system on the camera a translation vector of the camera coordinate system of the camera itself to the coordinate system of the calibration object;

[0118] S132, calculating the projection model of the camera in S131 and the initialized camera extrinsic parameter ; ;

[0119] S133, calculating the projection error of each corner point on the calibration board ; ;

[0120] S134, calculating the projection error of all corner points on the calibration board to obtain the projection error vector of all corner points , and then calculating the gradient and the Jacobian matrix ; is the first derivative of the projection error with respect to the camera extrinsic parameter ;

[0121] S135, calculating the update step ;

[0122] S136, then updating the camera extrinsic parameter according to the update step and the initialized camera extrinsic parameter to obtain the updated camera extrinsic parameter , and the calculation formula is specifically as follows:

[0123] ;

[0124] S137, repeating S132 to S136 until the error satisfies the predetermined requirement, and obtaining the pose estimation result of the zth frame of image, which is the updated camera extrinsic parameter;

[0125] S138, performing weighted average on the pose estimation results of all F frames of images of the camera to obtain the final camera extrinsic parameter of the camera ;

[0126] S139, repeating S131 to S138 until the final camera extrinsic parameter of all cameras in the camera array relative to the calibration object is obtained.

[0127] In some embodiments, the projection model of the camera in S131 is specifically as follows:

[0128] ;​​​​

[0129] in, Represents the theoretically calculated first... The camera at the The first observed in the frame The pixel coordinates of each corner point; Indicates camera The known intrinsic parameter matrix, Represents three-dimensional calibration points;

[0130] Projection error in S132 The formula for calculation is:

[0131] ;

[0132] in, Represents Euclidean distance; Indicates the actual calculated number of... The camera at the The first observed in the frame The theoretical pixel coordinates of each corner point.

[0133] In some embodiments, the intrinsic parameter matrix includes focal length, principal point, and distortion parameters.

[0134] In some embodiments, the Jacobian matrix in S133 is calculated as follows:

[0135] ;

[0136] in, The first Jacobian matrix represents the first Jacobian matrix. OK; This indicates the calculation of partial derivatives;

[0137] gradient in S134 The formula for calculation is:

[0138] ;

[0139] Where T represents the transpose of the matrix;

[0140] Update step size in S135 The formula for calculation is:

[0141] ;

[0142] in, It is a regularization parameter used to control the update step size. Size, It is the identity matrix. , It is a Hessian matrix.

[0143] In some embodiments, S4 specifically includes the following steps:

[0144] S21, by camera The projection model of the camera Camera coordinates The details are as follows:

[0145] (1)

[0146] in, , , For three-dimensional calibration points The three-dimensional coordinates; at the same time, since the calibration plate is assumed to be placed flat on the ground, And rotation matrix Translation vector The mathematical expression is:

[0147] (2)

[0148] (3)

[0149] In the formula, , , , , , , , , Representing rotation matrices respectively The parameters for each row and column; , , They represent translation vectors respectively. The translation amounts along the x, y, and z axes;

[0150] S22, Camera The camera coordinates are normalized to obtain the normalized camera coordinates, specifically:

[0151] (4)

[0152] in, , , These are the normalized camera coordinates;

[0153] S23. Substituting formula (4) into formulas (1) to (3), we obtain the following relationship:

[0154] (5)

[0155] (6)

[0156] (7)

[0157] S24, substitute formula (7) into formula (5) and formula (6) to obtain the following relationship:

[0158] (8)

[0159] (9)

[0160] S25, construct the camera coordinate system linear equation group by using formula (8) and formula (9), as follows:

[0161] (10)

[0162] S26, convert the camera coordinate system linear equation group into the following relationship:

[0163] (11)

[0164] wherein, and respectively represent two different intermediate parameters, and , ;

[0165] S27, decompose the intermediate parameter to obtain the following relationship:

[0166] (12)

[0167] wherein, and are orthogonal matrices, is a singular value diagonal matrix;

[0168] S28, calculate the pseudo-inverse of the intermediate parameter by using formula (12) and formula (13), as follows:

[0169] (13)

[0170] wherein, take the inverse of the non-zero singular value in , and the zero element remains zero during the inversion process;

[0171] S29, solve by pseudo-inverse to obtain the following relationship:

[0172] (14)

[0173] Then the formula (14) is used to solve the three-dimensional calibration point on the calibration object observed by the current camera.

[0174] In some embodiments, the S4 specifically comprises the following steps:

[0175] S41, for the same three-dimensional calibration point, selecting the observation results of at least three cameras, the observation results being the three-dimensional calibration point on the calibration object observed by the current camera; respectively backstepping the three-dimensional coordinates and taking the average to obtain the three-dimensional calibration point actual world coordinate mean ;

[0176] S42, in the camera array, selecting the corresponding three-dimensional calibration point in the to-be-detected camera, using the three-dimensional calibration point mapping and the three-dimensional calibration point actual world coordinate mean to calculate the Euclidean distance error , and the calculation formula is specifically as follows:

[0177] ;

[0178] S43, setting an error threshold , if the Euclidean distance error of the to-be-detected camera > the error threshold , the to-be-detected camera is determined as an abnormal camera, the positioning data output of the to-be-detected camera is suspended, and the calibration update is performed through S1 and S2, otherwise the calibration result of the to-be-detected camera is directly determined as correct.

[0179] The present application can dynamically eliminate the observation data of the abnormal camera through real-time detection of the Euclidean distance error, and trigger local re-calibration, which significantly improves the stability and anti-interference ability of the calibration parameters in the dynamic environment, and ensures the high-precision consistency of the geometric relationship of the multi-camera.

[0180] The present application converts the three-dimensional coordinate solving into efficient linear equation (i.e. camera coordinate system linear equation group) solving by establishing a linear algebra system based on normalized image coordinates (i.e. the steps S21 to S26), and in combination with the error threshold, the camera can be quickly detected, and the calculation complexity of online calibration is significantly reduced.

[0181] Another aspect of the present application also provides a camera array online cooperative calibration system for dynamic interference, comprising a camera array, and the camera array is calibrated by using the camera array online cooperative calibration method.

[0182] The above merely describes specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Furthermore, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the realization of the ordinary skilled person in the art, when the combination of the technical solutions appears contradictory or unachievable, it should be considered that the combination of the technical solutions does not exist, and is not within the protection scope required by the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for online collaborative calibration of camera arrays accommodating dynamic interference, characterized in that, Includes the following steps: S1. Solve for the camera extrinsic parameters of each camera in the camera array. The camera extrinsic parameters are the attitudes of each camera in the camera array relative to the calibration board. S2. Construct a system of linear equations for the camera coordinate system based on the camera's extrinsic parameters, and then solve the system of linear equations for the camera coordinate system to obtain the coordinates of a three-dimensional calibration point on the calibration object observed by the camera, which is the calibration result of the camera. S3, repeat S2, to obtain the coordinates of the three-dimensional calibration points on the calibration object observed by multiple cameras; S4. Calculate the mean of the actual world coordinates of the three-dimensional calibration points on the calibration object using the coordinates observed by multiple cameras. Then, calculate the Euclidean distance error of the camera to be tested based on the mean of the actual world coordinates of the three-dimensional calibration points. Determine whether the camera is abnormal based on the Euclidean distance error of the camera to be tested. If it is abnormal, update the calibration through S1 and S2. S1 specifically includes the following steps: S11. Place the calibration board within the overlapping field of view of the camera array. Each time the calibration board is moved or rotated, acquire one frame of image. Acquire a total of F frames of images to cover different poses. S12. For each frame of image from each camera in the camera array, the Harris corner detection algorithm is used to extract the pixel coordinates of all corners of the calibration board, so as to obtain the relationship between the three-dimensional geometric position of the corners on the calibration board and the corresponding point of the current corner in the image. S13. Solve the final camera extrinsic parameters of all cameras in the camera array relative to the calibration object based on the PnP algorithm; S2 specifically includes the following steps: S21, by camera The projection model of the camera Camera coordinates The details are as follows: （1） in, , , For three-dimensional calibration points The three-dimensional coordinates; at the same time, since the calibration plate is assumed to be placed flat on the ground, And rotation matrix Translation vector The mathematical expression is: （2） （3） In the formula, , , , , , , , , Representing rotation matrices respectively The parameters for each row and column; , , They represent translation vectors respectively. The translation amounts along the x, y, and z axes; S22, Camera The camera coordinates are normalized to obtain the normalized camera coordinates, specifically: （4） ； in, , , These are the normalized camera coordinates; S23. Substituting formula (4) into formulas (1) to (3), we obtain the following relationship: （5） （6） （7） S24. Substituting formula (7) into formulas (5) and (6), we obtain the following relationship: （8） （9） S25. Construct the linear equation system of the camera coordinate system using formulas (8) and (9), as follows: （10） S26. Transform the linear equations of the camera coordinate system into the following relation: （11） in, and These represent two different intermediate parameters, and , ; S27, intermediate parameters After decomposition, the following relation is obtained: （12） in, and It is an orthogonal matrix. It is a singular value diagonal matrix; S28. Calculate intermediate parameters using formulas (12) and (13). pseudo-reversal The details are as follows: （13） in, right The non-zero singular values ​​are taken in reverse order, and the zero element remains zero during the reciprocal process; S29, Through pseudo-inversion The following relationship is obtained by solving: （14） Then, the coordinates of the three-dimensional calibration point on the calibration object observed by the current camera are obtained by using formula (14).

2. The online collaborative calibration method for camera arrays accommodating dynamic interference as described in claim 1, characterized in that, S12 specifically includes the following steps: S121. First, establish the Harris response function for the z-th frame image, as follows: ； in, Represents the corner response value; det represents the determinant of the calculated matrix. Let z be the image gradient covariance matrix of the z-th frame; This represents calculating the trace of a matrix, which is the sum of the elements on the main diagonal. These are empirical constants; S122, then the diagonal response value With the image gradient covariance matrix Compare and filter out responses with values ​​greater than the image gradient covariance matrix. The points are identified, and duplicate points are removed using the non-maximum suppression method to obtain multiple corner points of the z-th frame image; S123, repeat S121 to S122 until multiple corresponding corner points are obtained for all F frames of images collected; S124. Create an initial index matrix with the same number of corner points as the calibration board; S125. Sort the detected corner points according to the geometry of the calibration plate. For each detected corner point, search for neighboring corner points along the boundary direction and expand the initial index matrix according to the distance relationship of neighboring corner points to obtain the expanded index matrix. S126. Then, find the pixel coordinates corresponding to the corner points on the calibration board on the amplified index matrix.

3. The online collaborative calibration method for camera arrays accommodating dynamic interference as described in claim 2, characterized in that, S13 specifically includes the following steps: S131. First, establish the cameras in the camera array. The projection model is then initialized, and the camera is initialized. Camera external parameters ,in Indicates camera The rotation matrix from the camera coordinate system to the calibration object coordinate system; Indicates camera The translation vector from the camera coordinate system to the calibration object coordinate system; S132, Based on the camera Projection model and initialized camera extrinsic parameters Calculate the projection error of each corner point on the calibration plate. ; S133, Calculate projection error Regarding the camera extrinsic parameters after initialization Jacobian matrix ; S134. Calculate the projection error of all corner points on the calibration board to obtain the projection error vector of all corner points. Then, based on the projection error vector of all corner points And Jacobi matrix Calculate gradient ,gradient It is projection error Regarding camera external parameters The first derivative; S135, Calculate the update step size ; S136, then based on the update step size and the camera extrinsic parameters after initialization Update camera extrinsic parameters to obtain updated camera extrinsic parameters. The specific calculation formula is as follows: ； S137, repeat S132 to S136 until error occurs. If the predetermined requirements are met, the pose estimation result of the z-th frame image is obtained, and the pose estimation result is the updated camera extrinsic parameters; S138, Camera The pose estimation results of all F frames are weighted and averaged to obtain the camera... Final camera extrinsic parameters; S139, repeat S131 to S138 until the final camera extrinsic parameters of all cameras in the camera array relative to the calibration object are obtained.

4. The online collaborative calibration method for camera arrays accommodating dynamic interference as described in claim 3, characterized in that, The camera array in S131 contains cameras The projection model is as follows: ； in, Represents the theoretically calculated first... The camera in the first The first observed in the frame The pixel coordinates of each corner point; Indicates camera The known intrinsic parameter matrix, Represents three-dimensional calibration points; Projection error in S132 The formula for calculation is: ； in, Represents Euclidean distance; Indicates the actual calculated number of... The camera in the first The first observed in the frame The theoretical pixel coordinates of each corner point.

5. The online collaborative calibration method for camera arrays accommodating dynamic interference according to claim 4, characterized in that, The intrinsic parameter matrix includes focal length, principal point, and distortion parameters.

6. The online collaborative calibration method for camera arrays accommodating dynamic interference according to claim 4, characterized in that, The formula for calculating the Jacobian matrix in S133 is as follows: ； in, The first Jacobian matrix represents the first Jacobian matrix. OK; This indicates the calculation of partial derivatives; gradient in S134 The formula for calculation is: ； Where T represents the transpose of the matrix; Update step size in S135 The formula for calculation is: ； in, It is a regularization parameter used to control the update step size. Size, It is the identity matrix. , It is a Hessian matrix.

7. The online collaborative calibration method for camera arrays accommodating dynamic interference according to claim 6, characterized in that, S4 specifically includes the following steps: S41. For the same 3D calibration point, select the observation results from at least three cameras. The observation results are the coordinates of the 3D calibration point on the calibration object observed by the current camera. After reversing the 3D coordinates of each camera, take the average to obtain the mean value of the actual world coordinates of the 3D calibration point. ; S42. In the camera array, select the corresponding 3D calibration point in the camera to be tested. Using the mapping of this 3D calibration point and the mean of the actual world coordinates of the 3D calibration point, calculate the Euclidean distance error between the two. The specific calculation formula is as follows: ； S43. Set error threshold If the Euclidean distance error of the camera to be tested Error threshold If the test camera is found to be abnormal, the output of its positioning data will be paused, and calibration will be updated via S1 and S2. Otherwise, the calibration result of the test camera will be considered correct.

8. An online collaborative calibration system for camera arrays resistant to dynamic interference, characterized in that, The camera array is calibrated using the online collaborative calibration method for camera arrays as described in any one of claims 1 to 7.

Images