Camera array calibration method, device and equipment based on optical center coordinate prior

Abstract

The invention relates to the technical field of camera calibration, and discloses a camera array calibration method, device and equipment based on optical center coordinate priori, and the method comprises the steps: obtaining a reference image collected by a target camera in a camera array, and extracting the two-dimensional coordinates of each preset reference point in a pixel coordinate system; obtaining the three-dimensional coordinates of each preset reference point in the world coordinate system, and constructing a conversion relation between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point; based on the conversion relation between the two-dimensional coordinates and the three-dimensional coordinates of the preset datum points, parameters of the target camera are determined through null-space analysis; and optimizing the parameters of the target camera by using a preset optimization method to obtain optimized parameters of the target camera. According to the invention, through conversion between the two-dimensional coordinates and the three-dimensional coordinates of the plurality of reference points, the camera parameters are determined through null-space analysis, calibration of the camera array without the common view field is realized, the efficiency, precision and stability of field calibration of the camera array are improved, and the measurement precision of a vision measurement system is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a camera array calibration method, device and equipment based on optical center coordinate prior. Background Art

[0002] Camera calibration is a key step to determine the three-dimensional information of the actual object from the image taken by the camera. It is usually achieved by taking multiple images using a calibration plane grid, that is, only coplanar points can be processed. For the camera calibration scene of the outdoor visual measurement system, the spatial points are often distributed in different planes, making it difficult to construct a coplanar configuration. Moreover, under long-distance observation conditions, the distribution of spatial points along the depth of field is much smaller than the observation distance, which is a singular configuration. Based on the above situation of outdoor scenes, it is not realistic to use large-size stereo targets or absolutely coplanar calibration plates for calibration or to implement self-calibration using self-motion constraints. In the calibration of multi-camera groups in the field, there is a certain degree of coupling between internal and external parameters, and as the focal length increases, the field of view angle will decrease rapidly, resulting in a significant weakening of the camera perspective imaging model, a sharp increase in the complex correlation between internal and external parameters, and an increase in the degree of coupling. The calibration problem of internal and external parameters of camera groups without a common field of view cannot be achieved. Summary of the invention

[0003] In view of this, the present invention provides a camera array calibration method, device and equipment based on optical center coordinate prior to solve the problem that it is impossible to calibrate the internal and external parameters of a camera array without a common field of view.

[0004] In a first aspect, the present invention provides a camera array calibration method based on optical center coordinate priors, wherein the camera array includes at least one camera, there is no common field of view between the cameras, and the field of view of each camera includes a plurality of non-collinear preset reference points, and the method includes: Obtain a reference image captured by a target camera in the camera array, and extract the two-dimensional coordinates of each preset reference point in the reference image in a pixel coordinate system. The reference images of each target camera are collected synchronously. Obtain the three-dimensional coordinates of each preset reference point in the world coordinate system, and use the camera perspective imaging model to construct a conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point; Based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point, the parameters of the target camera are determined using the optical center coordinate prior and null space analysis; The parameters of the target camera are optimized using a preset optimization method to obtain the optimized parameters of the target camera.

[0005] The camera array calibration method based on optical center coordinate prior provided by the present invention converts the two-dimensional coordinates and three-dimensional coordinates of multiple non-collinear reference points, determines the camera parameters by using null space analysis, and realizes the calibration of the camera array without a common field of view. It does not require a calibration plate and is more flexible. It improves the efficiency, accuracy and stability of on-site calibration of the camera array, thereby ensuring the measurement accuracy of the visual measurement system.

[0006] In an optional implementation, obtaining the three-dimensional coordinates of each preset reference point in the world coordinate system, and using the camera perspective imaging model to construct a conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point includes: Taking the total station setting coordinate system as the world coordinate system, using the total station to obtain the three-dimensional coordinates of each preset reference point in the total station setting coordinate system as the three-dimensional coordinates of each preset reference point in the world coordinate system; Based on the camera perspective imaging model, the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point is constructed:

[0007] in, is the depth factor, that is, the component of each control point in the z-axis direction in the camera coordinate system; is the homogeneous coordinate of the i-th point in the pixel coordinate system; is the three-dimensional coordinate of the i-th point in the world coordinate system; is the rotation matrix from the world coordinate system to the camera coordinate system, is the translation vector from the world coordinate system to the camera coordinate system; is the camera intrinsic parameter matrix:

[0008] in, , Respectively represent the equivalent focal length of the camera in the x-axis and y-axis directions, , is the coordinate of the camera principal point in the pixel coordinate system.

[0009] The camera array calibration method based on optical center coordinate prior provided by the present invention utilizes the total station setting coordinate system as the world coordinate system, and utilizes the total station to obtain the three-dimensional coordinates of each preset reference point. The three-dimensional coordinates are more accurate, and the acquisition method is simple, not restricted by the scene, and the application is more flexible.

[0010] In an optional implementation, based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point, the parameters of the target camera are determined using null space analysis, including: The optical center coordinates of the camera are estimated by marking points one by one; According to the coordinates of the optical center of the camera and the conversion relationship between the world coordinate system and the camera coordinate system, a matrix conversion formula for expressing the translation vector using a rotation matrix is ​​obtained; Preprocessing the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point to obtain a coordinate conversion formula that does not include a depth factor; Substitute the matrix transformation formula into the coordinate transformation formula, and perform a preset matrix transformation process to obtain a homogeneous equation group; The zero-space analysis is used to solve the aligned equations and obtain the parameters of the target camera.

[0011] The camera array calibration method based on optical center coordinate prior provided by the present invention reduces unknown quantities, reduces the difficulty of calculating camera parameters, and improves the camera calibration efficiency by converting the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point.

[0012] In an optional implementation, according to the optical center coordinates of the camera and based on the conversion relationship between the world coordinate system and the camera coordinate system, a matrix conversion formula for expressing the translation vector using a rotation matrix is ​​obtained, including: The conversion relationship between the world coordinate system and the camera coordinate system is:

[0013] in, Represents the coordinates of the preset reference point in the world coordinate system, Represents the camera coordinate system The coordinates of the corresponding preset reference points; According to the optical center coordinates of the camera, the matrix conversion formula of the translation vector represented by the rotation matrix is:

[0014] in, represents the optical center coordinates of the camera, , , is the row vector of the rotation matrix.

[0015] The camera array calibration method based on optical center coordinates prior provided by the present invention uses optical center coordinates to represent the translation vector with a rotation matrix, which reduces the unknown quantity in the translation vector, facilitates the rapid and accurate calculation of the values ​​of various parameters in the camera calibration process, and improves the calculation efficiency.

[0016] In an optional implementation, the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point is preprocessed to obtain a coordinate conversion formula that does not include a depth factor, including: Multiply both sides of the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point by , assuming ,get: ; Eliminate the depth factor on both sides and get the coordinate transformation formula: .

[0017] The camera array calibration method based on optical center coordinate prior provided by the present invention eliminates the unknown quantity of depth factor by preprocessing the transformation relationship between the two-dimensional coordinates and the three-dimensional coordinates of each reference point, and converts the matrix calculation into an algebraic expression for calculation, thereby reducing the calculation difficulty.

[0018] In an optional implementation, the matrix transformation formula is substituted into the coordinate transformation formula, and a preset matrix transformation process is performed to obtain a homogeneous equation group, including: Substitute the matrix transformation formula into the coordinate transformation formula, shift the terms, merge them, and divide both sides by the equivalent focal length. ,get: ; Define two preset matrices:

[0019]

[0020] in, Indicates preset reference points. Multiple preset reference points can obtain multiple overdetermined equations. , then we get the homogeneous system of equations .

[0021] The camera array calibration method based on optical center coordinate prior provided by the present invention defines two preset matrices and combines the matrix transformation formula and the coordinate transformation formula to obtain a homogeneous equation group, and uses null space analysis to solve the homogeneous equation group. The solution of camera parameters is mainly achieved by mathematical means, which reduces the dependence of the camera calibration process on the scene and has stronger applicability.

[0022] In an optional implementation, optimizing the parameters of the target camera using a preset optimization method to obtain the optimized parameters of the target camera includes: The internal and external parameters of the camera are solved by using bundle adjustment optimization. The optimization formula under single field of view is:

[0023] Wherein, n represents the number of preset reference points.

[0024] The camera array calibration method based on optical center coordinate prior provided by the present invention optimizes the parameters of the camera by using a preset optimization method, thereby improving the precision of camera calibration and the accuracy of camera imaging.

[0025] In a second aspect, the present invention provides a camera array calibration device based on optical center coordinate priors, wherein the camera array includes at least one camera, there is no common field of view between the cameras, and the field of view of each camera includes a plurality of non-collinear preset reference points, and the device includes: A two-dimensional coordinate determination module is used to obtain a reference image captured by a target camera in the camera array, and extract the two-dimensional coordinates of each preset reference point in the reference image in a pixel coordinate system. The reference images of each target camera are collected synchronously. A coordinate conversion relationship determination module is used to obtain the three-dimensional coordinates of each preset reference point in the world coordinate system, and to construct a conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point using a camera perspective imaging model; A parameter determination module is used to determine the parameters of the target camera based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point by using the optical center coordinate prior and null space analysis; The parameter optimization module is used to optimize the parameters of the target camera using a preset optimization method to obtain the optimized parameters of the target camera.

[0026] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0027] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0028] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 is a schematic flow chart of a camera array calibration method based on optical center coordinate prior according to an embodiment of the present invention; Figure 2is a flow chart of another camera array calibration method based on optical center coordinate prior according to an embodiment of the present invention; Figure 3 is a structural block diagram of a camera array calibration device based on optical center coordinate prior according to an embodiment of the present invention; Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0032] An embodiment of the present invention provides a camera array calibration method based on optical center coordinate priors, which converts the two-dimensional coordinates of multiple non-collinear reference points into three-dimensional coordinates and uses null space analysis to determine camera parameters, so as to achieve the effect of calibrating the internal and external parameters of a camera array without a common field of view.

[0033] According to an embodiment of the present invention, an embodiment of a camera array calibration method based on optical center coordinate prior is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] In this embodiment, a camera array calibration method based on optical center coordinate prior is provided, which can be used in the above-mentioned computer system. Figure 1 is a flow chart of a camera array calibration method based on optical center coordinate prior according to an embodiment of the present invention, wherein the camera array includes at least one camera, there is no common field of view between the cameras, and the field of view of each camera includes a plurality of non-collinear preset reference points, such as Figure 1 As shown, the process includes the following steps: Step S101 , obtaining a reference image captured by a target camera in a camera array, and extracting the two-dimensional coordinates of each preset reference point in the reference image in a pixel coordinate system. The reference images of each target camera are captured synchronously.

[0035] Specifically, for outdoor visual measurement systems, in order to ensure the integrity of the captured images, visual measurement systems often require multiple cameras to form a camera array to cooperate, and the calibration of multiple cameras in the camera array must be unified to achieve the measurement of the complete outdoor scene. If the shooting field of view of a single camera can cover the scene to be measured, the camera array includes one camera, and only a single camera needs to be calibrated. In this embodiment, there is no common field of view between the cameras that make up the camera array, and each camera is calibrated separately. The multiple cameras are linked together by the position of each camera in the world coordinate system to achieve the calibration of the entire camera array.

[0036] The target camera is used to shoot to obtain a reference image, and the two-dimensional coordinates of each preset reference point in the reference image are extracted in the pixel coordinate system. It should be noted that the field of view of each target camera includes at least 5 preset reference points, and the preset reference points are not collinear. Each target camera needs to shoot at the same time, collect its own reference image, and keep the shooting time consistent, which can avoid deviations in camera position and multiple reference point positions of each target camera caused by different times, and improve the availability of multiple reference images and the accuracy of camera array calibration.

[0037] Step S102, obtaining the three-dimensional coordinates of each preset reference point in the world coordinate system, and constructing a conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point using a camera perspective imaging model.

[0038] Specifically, the world coordinate system can be any world coordinate system established according to actual conditions, and the three-dimensional coordinates of each preset reference point in the world coordinate system are measured and determined using existing mature measurement methods. The camera perspective imaging model, also known as the pinhole camera model, is a simplified optical imaging model that uses central perspective projection to project objects in three-dimensional space (world coordinate system) through a pinhole onto a two-dimensional imaging plane (pixel coordinate system). This model ignores complex factors such as lens distortion and light diffraction, making calculation and analysis relatively simple. Therefore, the camera perspective imaging model can be used to construct the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point.

[0039] Step S103 : Based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point, the parameters of the target camera are determined by using null space analysis.

[0040] Specifically, based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point, a set of equations is obtained through mathematical conversion and calculation, and the null space analysis method is used to perform high-precision linear solution to obtain the parameters of the target camera.

[0041] Step S104: Optimize the parameters of the target camera using a preset optimization method to obtain the optimized parameters of the target camera.

[0042] Specifically, the preset optimization method can be any existing mature technology that can optimize camera parameters, which will not be described in detail here. The optimized camera parameters are conducive to improving the accuracy of camera calibration.

[0043] The camera array calibration method based on optical center coordinate prior provided in this embodiment converts the two-dimensional coordinates and three-dimensional coordinates of multiple non-collinear reference points, determines the camera parameters by using null space analysis, and realizes the calibration of the camera array without a common field of view. It does not require a calibration plate and is more flexible. It improves the efficiency, accuracy and stability of on-site calibration of the camera array, thereby ensuring the measurement accuracy of the visual measurement system.

[0044] In this embodiment, a camera array calibration method based on optical center coordinate prior is provided, which can be used in the above-mentioned computer system. Figure 2 is a flow chart of a camera array calibration method based on optical center coordinate prior according to an embodiment of the present invention, wherein the camera array includes at least one camera, there is no common field of view between the cameras, and the field of view of each camera includes a plurality of non-collinear preset reference points, such as Figure 2 As shown, the process includes the following steps: Step S201, obtain the reference image captured by the target camera in the camera array, and extract the two-dimensional coordinates of each preset reference point in the reference image in the pixel coordinate system. The reference images of each target camera are collected synchronously. For details, please refer to Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0045] Step S202, obtaining the three-dimensional coordinates of each preset reference point in the world coordinate system, and constructing a conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point using a camera perspective imaging model.

[0046] Specifically, the above step S202 includes: Step S2021, taking the total station setup coordinate system as the world coordinate system, using the total station to obtain the three-dimensional coordinates of each preset reference point in the total station setup coordinate system as the three-dimensional coordinates of each preset reference point in the world coordinate system.

[0047] Specifically, the total station is a commonly used instrument for outdoor measurement. The three-dimensional coordinate measurement accuracy of the total station can reach millimeters or even sub-millimeter levels. Using the total station's station coordinate system as the world coordinate system can improve the accuracy of camera calibration. Using the total station's station coordinate system as the world coordinate system, the total station can be used to directly measure the three-dimensional coordinates of each preset reference point.

[0048] Step S2022: construct a conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point based on the camera perspective imaging model: (1) in, is the depth factor, that is, the component of each control point in the z-axis direction in the camera coordinate system; is the homogeneous coordinate of the i-th point in the pixel coordinate system; is the three-dimensional coordinate of the i-th point in the world coordinate system; is the rotation matrix from the world coordinate system to the camera coordinate system, is the translation vector from the world coordinate system to the camera coordinate system; is the camera intrinsic parameter matrix: (2) in, , Respectively represent the equivalent focal length of the camera in the x-axis and y-axis directions, , is the coordinate of the camera principal point in the pixel coordinate system.

[0049] Specifically, the homogeneous coordinates of the i-th point in the pixel coordinate system are , the three-dimensional coordinates of the i-th point in the world coordinate system are .

[0050] is the camera intrinsic parameter matrix, , They represent the equivalent focal lengths of the camera in the x-axis and y-axis directions, respectively. Since the pixel size is generally a square, the equivalent focal lengths in the x-axis and y-axis directions can be considered equal, that is, . , It is the coordinate of the camera principal point in the pixel coordinate system, that is, the coordinate of the camera optical center in the pixel coordinate system. In practical applications, it can be assumed that the image center is used as the principal point coordinate to facilitate the determination of the principal point coordinate without losing measurement accuracy. Alternatively, a short-focus lens can be used instead of a long-focus lens for pre-calibration. This process is a mature existing technology and will not be repeated here. Therefore, the camera internal parameters can be simplified to include only the equivalent focal length This is the form of an unknown parameter.

[0051] is the rotation matrix from the world coordinate system to the camera coordinate system, , you can set: (3) in, Represents the row vector of the rotation matrix.

[0052] is the translation vector from the world coordinate system to the camera coordinate system, , , that is, the coordinates of the origin of the world coordinate system. After rotation by the rotation matrix and translation by the mean vector, the coordinates of the origin in the camera coordinate system are obtained.

[0053] The camera array calibration method based on optical center coordinate prior provided in this embodiment uses the total station setting coordinate system as the world coordinate system, and uses the total station to obtain the three-dimensional coordinates of each preset reference point. The three-dimensional coordinates are more accurate, and the acquisition method is simple, not restricted by the scene, and more flexible in application.

[0054] Step S203 , based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point, the parameters of the target camera are determined by using the optical center coordinate prior and null space analysis.

[0055] Specifically, the above step S203 includes: Step S2031, obtaining the optical center coordinates of the camera by estimating the points one by one.

[0056] Specifically, relevant technologies show that the three-dimensional position coordinate measurement error of the camera's optical center at the decimeter level or even the meter level will not have a sensitive impact on the camera calibration. The three-dimensional coordinate measurement accuracy of the total station can reach the millimeter level or even the sub-millimeter level. Therefore, the total station's station coordinate system is used as the world coordinate system. By marking the camera one by one, the rough three-dimensional position coordinates of the camera's optical center can be obtained. The specific process is a mature existing technology and will not be repeated here.

[0057] Step S2032, according to the optical center coordinates of the camera and based on the conversion relationship between the world coordinate system and the camera coordinate system, a matrix conversion formula for expressing the translation vector using a rotation matrix is ​​obtained.

[0058] In some optional implementations, the above step S2032 includes: Step a1: The conversion relationship between the world coordinate system and the camera coordinate system is: (4) in, Represents the coordinates of the preset reference point in the world coordinate system, Represents the camera coordinate system The coordinates of the corresponding preset reference point.

[0059] Specifically, , , , are all in vector form, where It is a matrix composed of the three-dimensional coordinates of multiple preset reference points in the world coordinate system. , is the camera coordinate system The corresponding three-dimensional coordinates of the preset reference point, including the camera internal parameters, are multiplied by the left and right sides of formula (1) ,get .

[0060] Step a2, according to the optical center coordinates of the camera, the matrix conversion formula of the translation vector expressed by the rotation matrix is: (5) in, represents the optical center coordinates of the camera, , , is the row vector of the rotation matrix.

[0061] Specifically, according to formula (4), Represents the coordinates of the optical center of the camera, so we get formula (5). By directly measuring the three-dimensional coordinates of the optical center in the world coordinate system through the total station, the translation vector in the camera extrinsic parameters can be represented by a rotation matrix to realize the optical center coordinate prior.

[0062] The camera array calibration method based on optical center coordinates prior provided in this embodiment uses optical center coordinates to represent the translation vector with a rotation matrix, thereby reducing the unknown quantities in the translation vector, facilitating fast and accurate calculation of the values ​​of various parameters in the camera calibration process, and improving calculation efficiency.

[0063] Step S2033: pre-process the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point to obtain a coordinate conversion formula that does not include a depth factor.

[0064] In some optional implementations, the above step S2033 includes: Step b1: multiply both sides of the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point by , assuming ,get: (6) Step b2, eliminate the depth factor on both sides and get the coordinate transformation formula: (7).

[0065] Specifically, formula (6) contains three equations: , , , Substituting the third equation into the first two equations, the depth factor can be eliminated and the coordinate transformation formula can be obtained. From formula (7), it can be seen that the focal length is coupled with the rotation matrix and the translation vector. If the translation vector can be represented by the rotation matrix, the number of mutually coupled unknown quantities to be measured can be reduced by half.

[0066] The camera array calibration method based on optical center coordinate prior provided by the present invention eliminates the unknown quantity of depth factor by preprocessing the transformation relationship between the two-dimensional coordinates and the three-dimensional coordinates of each reference point, and converts the matrix calculation into an algebraic expression for calculation, thereby reducing the calculation difficulty.

[0067] Step S2034, substitute the matrix transformation formula into the coordinate transformation formula, and perform a preset matrix transformation process to obtain a homogeneous equation group.

[0068] In some optional implementations, the above step S2034 includes: Step c1, substitute the matrix transformation formula (5) into the coordinate transformation formula (7), and perform the shifting and merging, and then divide both sides by the equivalent focal length. ,get: (8).

[0069] Step c2, define two preset matrices: (9) (10) in, Indicates preset reference points. Multiple preset reference points can obtain multiple overdetermined equations. , then we get the homogeneous system of equations .

[0070] Specifically, two preset matrices are defined , , the purpose is to write formula (8) as In the form of matrix multiplication, formula (8) is written as:

[0071] According to the above formula, two preset matrices (Formula (9) and Formula (10) are defined. For each preset reference point, its corresponding There are two equations. N preset reference points can get 2N equations. Let: , then the homogeneous system of equations consisting of 2N equations can be written as: .

[0072] The camera array calibration method based on optical center coordinate prior provided in this embodiment defines two preset matrices and combines the matrix transformation formula and the coordinate transformation formula to obtain a homogeneous equation group, and uses null space analysis to solve the homogeneous equation group. The solution of camera parameters is mainly achieved by mathematical means, which reduces the dependence of the camera calibration process on the scene and has stronger applicability.

[0073] Step S2035, using null space analysis to solve the alignment equations to obtain the parameters of the target camera.

[0074] Specifically, the null space analysis method is used to perform high-precision linear solutions to the aligned equations. If the orthogonality of the rotation matrix is ​​not considered, due to the preset matrix The dimension is 9, so at least 9 equations are needed to solve, that is, 2N=9, then N>4.5. Since N is an integer, at least 5 homogeneous equations obtained by preset reference points are required to solve the preset matrix , obtained by solving the linear equations, The initial values ​​of the rotation matrix and focal length are decomposed and then the initial value of the translation vector is calculated using formula (5).

[0075] The camera array calibration method based on optical center coordinate prior provided in this embodiment reduces unknown quantities, reduces the difficulty of calculating camera parameters, and improves the efficiency of camera calibration by converting the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point.

[0076] Step S204: Optimize the parameters of the target camera using a preset optimization method to obtain the optimized parameters of the target camera.

[0077] Specifically, the bundle adjustment is used to optimize the internal and external parameters of the camera. The optimization formula under single field of view is:

[0078] Wherein, n represents the number of preset reference points.

[0079] Specifically, based on the linear solution of the camera, the initial values ​​of the camera's internal and external parameters are obtained, and the camera's internal and external parameters are solved using bundle adjustment optimization according to the imaging relationship between each preset reference point and the corresponding image point. Through this embodiment, the internal parameters of each target camera in the camera array and the posture relationship between the total station setting coordinate system can be obtained. Further, the posture relationship between each camera can be obtained by coordinate system transformation, thereby realizing the calibration process of the camera array.

[0080] The camera array calibration method based on optical center coordinate prior provided in this embodiment optimizes the parameters of the camera using a preset optimization method, thereby improving the precision of camera calibration and the accuracy of camera imaging.

[0081] The camera array calibration method based on optical center coordinate prior provided in this embodiment converts the two-dimensional coordinates and three-dimensional coordinates of multiple non-collinear reference points, determines the camera parameters by using null space analysis, and realizes the calibration of the camera array without a common field of view. It does not require a calibration plate and is more flexible. It improves the efficiency, accuracy and stability of on-site calibration of the camera array, thereby ensuring the measurement accuracy of the visual measurement system.

[0082] The above embodiment is the calibration of a single camera, that is, obtaining the intrinsic and extrinsic parameters of the target camera. For a camera array, since the spatial position relationship between the cameras is fixed, it is only necessary to ensure that all cameras are synchronously capturing images, and then use their respective images to calculate the intrinsic and extrinsic parameters of each camera to achieve the calibration of the camera array.

[0083] In this embodiment, a camera array calibration device based on optical center coordinate prior is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0084] This embodiment provides a camera array calibration device based on optical center coordinate priors. The camera array includes at least one camera. There is no common field of view between the cameras. The field of view of each camera includes multiple non-collinear preset reference points, such as Figure 3 As shown, including: The two-dimensional coordinate determination module 301 is used to obtain the reference image captured by the target camera in the camera array and extract the two-dimensional coordinates of each preset reference point in the reference image in the pixel coordinate system. The reference images of each target camera are collected synchronously.

[0085] The coordinate conversion relationship determination module 302 is used to obtain the three-dimensional coordinates of each preset reference point in the world coordinate system, and construct the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point using the camera perspective imaging model.

[0086] The parameter determination module 303 is used to determine the parameters of the target camera based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point by using the optical center coordinate prior and null space analysis.

[0087] The parameter optimization module 304 is used to optimize the parameters of the target camera using a preset optimization method to obtain the optimized parameters of the target camera.

[0088] In some optional implementations, the coordinate transformation relationship determination module 302 includes: The world coordinate system establishment unit is used to use the total station setting coordinate system as the world coordinate system, and use the total station to obtain the three-dimensional coordinates of each preset reference point in the total station setting coordinate system as the three-dimensional coordinates of each preset reference point in the world coordinate system.

[0089] The conversion relationship determination unit is used to construct the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point based on the camera perspective imaging model.

[0090] In some optional implementations, the parameter determination module 303 includes: The optical center coordinate estimation unit is used to estimate the optical center coordinates of the camera by marking points one by one.

[0091] The matrix conversion determination unit is used to obtain a matrix conversion formula for expressing a translation vector using a rotation matrix according to the optical center coordinates of the camera and based on the conversion relationship between the world coordinate system and the camera coordinate system.

[0092] The coordinate conversion determining unit is used to pre-process the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point to obtain a coordinate conversion formula that does not include a depth factor.

[0093] The matrix conversion preprocessing unit is used to substitute the matrix conversion formula into the coordinate conversion formula and perform a preset matrix conversion process to obtain a homogeneous equation group.

[0094] The parameter solving unit is used to solve the aligned equations by using null space analysis to obtain the parameters of the target camera.

[0095] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0096] The camera array calibration device based on optical center coordinate prior in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0097] The embodiment of the present invention also provides a computer device having the above Figure 3 The camera array calibration device based on optical center coordinate prior is shown.

[0098] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0099] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0100] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0101] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0102] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0103] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0104] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0105] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A camera array calibration method based on optical center coordinate prior, characterized in that: The camera array includes at least one camera, there is no common field of view between the cameras, and the field of view of each camera includes a plurality of non-collinear preset reference points. The method includes: Acquire a reference image captured by a target camera in the camera array, and extract the two-dimensional coordinates of each preset reference point in the reference image in a pixel coordinate system, wherein the reference images of each target camera are captured synchronously; Obtain the three-dimensional coordinates of each preset reference point in the world coordinate system, and use the camera perspective imaging model to construct a conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point; Based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point, the parameters of the target camera are determined by using the optical center coordinate prior and null space analysis; The parameters of the target camera are optimized using a preset optimization method to obtain the optimized parameters of the target camera.

2. The method according to claim 1, characterized in that Obtain the three-dimensional coordinates of each preset reference point in the world coordinate system, and use the camera perspective imaging model to construct the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point, including: Taking the total station setting coordinate system as the world coordinate system, using the total station to obtain the three-dimensional coordinates of each preset reference point in the total station setting coordinate system as the three-dimensional coordinates of each preset reference point in the world coordinate system; Based on the camera perspective imaging model, the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point is constructed: in, is the depth factor, that is, the component of each control point in the z-axis direction in the camera coordinate system; is the homogeneous coordinate of the i-th point in the pixel coordinate system; is the three-dimensional coordinate of the i-th point in the world coordinate system; is the rotation matrix from the world coordinate system to the camera coordinate system, is the translation vector from the world coordinate system to the camera coordinate system; is the camera intrinsic parameter matrix: in, , Respectively represent the equivalent focal length of the camera in the x-axis and y-axis directions, , is the coordinate of the camera principal point in the pixel coordinate system.

3. The method according to claim 2, characterized in that Based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point, the parameters of the target camera are determined by using the optical center coordinate prior and null space analysis, including: The optical center coordinates of the camera are estimated by marking points one by one; According to the optical center coordinates of the camera and based on the conversion relationship between the world coordinate system and the camera coordinate system, a matrix conversion formula for expressing the translation vector using the rotation matrix is ​​obtained; Preprocessing the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point to obtain a coordinate conversion formula that does not include a depth factor; Substituting the matrix transformation formula into the coordinate transformation formula, and performing a preset matrix transformation process to obtain a homogeneous equation group; The homogeneous equations are solved using null space analysis to obtain the parameters of the target camera.

4. The method according to claim 3, characterized in that According to the optical center coordinates of the camera and based on the conversion relationship between the world coordinate system and the camera coordinate system, a matrix conversion formula for expressing the translation vector using the rotation matrix is ​​obtained, including: The conversion relationship between the world coordinate system and the camera coordinate system is: in, Represents the coordinates of the preset reference point in the world coordinate system, Represents the camera coordinate system The coordinates of the corresponding preset reference points; According to the optical center coordinates of the camera, the matrix conversion formula for expressing the translation vector using the rotation matrix is: in, represents the optical center coordinates of the camera, , , is the row vector of the rotation matrix.

5. The method according to claim 4, characterized in that The conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point is preprocessed to obtain a coordinate conversion formula that does not include a depth factor, including: Multiply both sides of the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point by , assuming ,get: ； Eliminate the depth factor on both sides and get the coordinate transformation formula: 。 6. The method according to claim 5, characterized in that Substitute the matrix transformation formula into the coordinate transformation formula, and perform a preset matrix transformation process to obtain a homogeneous equation group, including: Substitute the matrix transformation formula into the coordinate transformation formula, and perform transposition and merging, and divide both sides by the equivalent focal length. ,get: ； Define two preset matrices: in, Indicates preset reference points. Multiple preset reference points can obtain multiple overdetermined equations. , then we get the homogeneous system of equations .

7. The method according to claim 1, characterized in that Optimizing the parameters of the target camera using a preset optimization method to obtain the optimized parameters of the target camera includes: The internal and external parameters of the camera are solved by using bundle adjustment optimization. The optimization formula under single field of view is: Wherein, n represents the number of preset reference points.

8. A camera array calibration device based on optical center coordinate prior, characterized in that: The camera array includes at least one camera, there is no common field of view between the cameras, and the field of view of each camera includes a plurality of non-collinear preset reference points, and the device includes: A two-dimensional coordinate determination module, used to obtain a reference image captured by a target camera in the camera array, and extract the two-dimensional coordinates of each preset reference point in the reference image in a pixel coordinate system, wherein the reference images of each target camera are captured synchronously; A coordinate conversion relationship determination module is used to obtain the three-dimensional coordinates of each preset reference point in the world coordinate system, and to construct a conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of each preset reference point using a camera perspective imaging model; A parameter determination module, for determining the parameters of the target camera based on the conversion relationship between the two-dimensional coordinates and the three-dimensional coordinates of the preset reference points by using the optical center coordinate prior and null space analysis; The parameter optimization module is used to optimize the parameters of the target camera using a preset optimization method to obtain the optimized parameters of the target camera.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images