Three-camera-based Color 3D Imaging System and Its Calibration Method

Abstract

The present invention provides a color three-dimensional imaging system based on three cameras and a calibration method therefor. The method includes: respectively collecting a first image, a second image and a third image containing a target by a left camera, a right camera and a color camera, and obtaining the pixel coordinates of the fiducial points in the first image, the second image and the third image; calibrating the binocular system formed by the left and right cameras according to the pixel coordinates of the fiducial points in the first and second images by using the backward projection principle of the left and right cameras to obtain first calibration parameters; calibrating the color camera according to the external parameter matrix of the left camera and the pixel coordinates of the fiducial points in the third image by using the forward projection principle of the color camera to obtain second calibration parameters. The present invention completes the calibration of the binocular system by reconstructing the imaging model of the left and right cameras through backward projection; and completes the calibration of the color camera by reconstructing the imaging model of the color camera by using forward projection, so that the calibration optimization objective function is consistent with the color three-dimensional reconstruction process, and the calibration accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional measurement, and particularly relates to a color three-dimensional imaging system based on three cameras and a calibration method thereof.

Background Art

[0002] Camera calibration is an essential step in image measurement or machine vision applications. The accuracy of its calibration results and the stability of the algorithm directly affect the accuracy of the camera's working results. Usually, the calibration method of a color three-dimensional imaging system based on three cameras is implemented based on two grayscale cameras and one color camera: the left and right grayscale cameras form a certain angle to form a binocular stereo imaging system. The three-dimensional reconstruction of the binocular stereo imaging system is completed by finding the corresponding points of the left and right cameras and combining the structural parameters of the binocular stereo imaging. Then, according to the coordinate transformation relationship between the color camera and the reconstructed three-dimensional model and the imaging parameters of the color camera, the projection image coordinates of the three-dimensional model in the color camera are obtained, so as to endow the reconstructed three-dimensional model with color information.

[0003] However, in the prior art, the calibration process of a color three-dimensional imaging system based on three cameras mostly uses the error between the actual value and the theoretical value of the three-camera image coordinates as the optimization objective function. This calibration method does not consider the error of the reconstructed three-dimensional model projected in the color camera, that is, the distance between the projection position of the reconstructed three-dimensional model obtained through the imaging of the color camera and the image position of the true imaging, resulting in a large error and low accuracy in the final calibration result. In view of this, it is necessary to provide a color three-dimensional imaging system based on three cameras and a calibration method thereof to overcome the above defects.

Summary of the Invention

[0004] The present invention proposes a color three-dimensional imaging system based on three cameras and a calibration method thereof that can improve the calibration accuracy.

[0005] To achieve the above object, the present invention provides a calibration method for a color three-dimensional imaging system based on three cameras, and the method includes the following steps:

[0006] Establish a color three-dimensional imaging system including two grayscale cameras and one color camera, wherein the two grayscale cameras are composed of a left camera and a right camera;

[0007] Collect a first image, a second image, and a third image containing a target through the left camera, the right camera, and the color camera respectively, and obtain the pixel coordinates of the fiducial points in the first image, the second image, and the third image respectively; wherein a plurality of fiducial points are provided on the target;

[0008] Calibrate the binocular system composed of the left and right cameras based on the pixel coordinates of the fiducial points in the first image and the second image using the back-projection principle of the left and right cameras to obtain the first calibration parameters of the binocular system. The first calibration parameters at least include the external parameter matrix of the left camera, the transformation parameters between the left and right cameras, the internal parameters, and the distortion parameters.

[0009] Calibrate the color camera based on the external parameter matrix of the left camera and the pixel coordinates of the fiducial points in the third image using the forward-projection principle of the color camera to obtain the second calibration parameters. The second calibration parameters at least include the transformation parameters corresponding to the color camera and the left camera, the internal parameters of the color camera, and the distortion parameters.

[0010] In a preferred embodiment, the step of calibrating the binocular system composed of the left and right cameras based on the pixel coordinates of the fiducial points in the first image and the second image using the back-projection principle of the left and right cameras to obtain the first calibration parameters of the binocular system includes:

[0011] Establish a linear projection model of the left and right cameras based on the pixel coordinates of the fiducial points in the first image and the second image using the distortion model of back-projection.

[0012] Obtain the observed values of the three-dimensional coordinates of the fiducial points according to the linear projection model of the left and right cameras using the ray intersection principle of triangulation.

[0013] Construct a first optimization objective function based on the Euclidean distance deviation between the observed values and the actual values of the three-dimensional coordinates of the fiducial points, and minimize the first optimization objective function through iterative optimization calculations to obtain the first calibration parameters.

[0014] In a preferred embodiment, the first optimization objective function includes the first calibration parameters to be optimized. The distortion parameters of the left and right cameras respectively include the radial distortion parameters and tangential distortion parameters corresponding to the left camera, and the radial distortion parameters and tangential distortion parameters corresponding to the right camera.

[0015] In a preferred embodiment, the step of establishing a linear projection model of the left and right cameras based on the pixel coordinates of the fiducial points in the first image and the second image using the distortion model of back-projection includes:

[0016] Calculate the relational expression for transforming from pixel coordinates to distorted image coordinates as the first transformation relational expression, and calculate the relational expression for transforming from distorted image coordinates to undistorted image coordinates as the second transformation relational expression. The first transformation relational expression contains the internal parameters of the left camera or the right camera, and the second transformation relational expression contains the distortion parameters of the left camera or the right camera.

[0017] Substitute the pixel coordinates of the fiducial points in the first image and the second image into the first transformation relation and the second transformation relation respectively to obtain the linear projection models of the left and right cameras.

[0018] In a preferred embodiment, the step of calibrating the color camera according to the external parameter matrix of the left camera and the pixel coordinates of the fiducial points in the third image by using the forward projection principle of the color camera to obtain the second calibration parameters includes:

[0019] Transform the world coordinate system into the camera coordinate system corresponding to the left camera according to the external parameter matrix of the left camera, and establish the transformation relationship between the camera coordinate system corresponding to the left camera and the camera coordinate system corresponding to the color camera to obtain the expression of the three-dimensional coordinates of the fiducial points in the camera coordinate system corresponding to the color camera;

[0020] According to the expression of the three-dimensional coordinates of the fiducial points in the camera coordinate system corresponding to the color camera, use the internal parameters of the color camera and the distortion model of forward projection to obtain the theoretical value of the pixel coordinates corresponding to the fiducial points in the projection image of the color camera;

[0021] Construct a second optimization objective function based on the Euclidean distance deviation between the theoretical value of the pixel coordinates of the fiducial points and the actual pixel coordinates of the corresponding fiducial points in the third image, and minimize the second optimization objective function through iterative optimization calculation to obtain the second calibration parameters.

[0022] In a preferred embodiment, the second optimization objective function includes the second calibration parameters to be optimized, and the distortion parameters of the color camera include the radial distortion parameters and tangential distortion parameters corresponding to the color camera.

[0023] In a preferred embodiment, the step of using the internal parameters of the color camera and the distortion model of forward projection to obtain the theoretical value of the pixel coordinates corresponding to the fiducial points in the projection image of the color camera according to the expression of the three-dimensional coordinates of the fiducial points in the camera coordinate system corresponding to the color camera includes:

[0024] Obtain the normalized image coordinates of the fiducial points in the projection image of the color camera according to the expression of the three-dimensional coordinates of the fiducial points in the camera coordinate system corresponding to the color camera;

[0025] Calculate the relation for transforming the normalized image coordinates of the fiducial points in the projection image of the color camera to the distorted image coordinates as the third transformation relation, and the third transformation relation contains the distortion parameters of the color camera;

[0026] Calculate the relationship for transforming the distorted image coordinates in the third transformation relationship to the theoretical pixel coordinates as the fourth transformation relationship, where the fourth transformation relationship includes the internal parameters of the color camera.

[0027] Obtain the calculation expression of the theoretical pixel coordinates according to the third transformation relationship and the fourth transformation relationship, and use it as the theoretical value of the pixel coordinates.

[0028] In a preferred embodiment, place the target at different angles and positions within the common effective area of the three cameras, so that the left camera, the right camera, and the color camera respectively collect the first image, the second image, and the third image containing the target at different positions.

[0029] To achieve the above object, the present invention also provides a color three-dimensional imaging system based on three cameras, including two grayscale cameras and a color camera. The two grayscale cameras are composed of a left camera and a right camera. The left camera, the right camera, and the color camera are used to respectively collect the first image, the second image, and the third image containing the target, and a plurality of fiducial points are arranged on the target; the system also includes a memory and a processor connected to the left camera, the right camera, and the color camera; the memory is used to store a computer program that can run on the processor, and when the processor executes the computer program, it implements the calibration method of the above-mentioned color three-dimensional imaging system based on three cameras.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows: By using the above-mentioned color three-dimensional imaging system based on three cameras and its calibration method, in the specific calibration process, first use the back-projection principle to reconstruct the imaging model of the binocular system including the left and right cameras, which is beneficial to directly remove distortion from the image and complete the calibration of the binocular system; then use the forward-projection principle to reconstruct the imaging model of the color camera and complete the calibration of the color camera, making the calibration optimization objective function consistent with the color three-dimensional reconstruction process, which is beneficial to improving the calibration accuracy.

[0031] To make the above objects, features, and advantages of the invention more obvious and understandable, the following specifically lists the preferred embodiments of the invention and, in conjunction with the accompanying drawings, makes a detailed description as follows.

Description of the Drawings

[0032] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other relevant drawings can also be obtained based on these drawings.

[0033] Figure 1Schematic diagram of the corresponding relationship between the left camera, the right camera and the color camera in an embodiment of the present invention;

[0034] Figure 2 Flowchart of a calibration method for a color three-dimensional imaging system based on three cameras provided by a preferred embodiment of the present invention;

[0035] Figure 3 is Figure 2 Sub-flowchart of step S203 shown in

[0036] Figure 4 is Figure 2 Sub-flowchart of step S204 shown in

[0037] Figure 5 Structural block diagram of a color three-dimensional imaging system based on three cameras provided by a preferred embodiment of the present invention.

Detailed implementation manners

[0038] In order to make the objectives, technical solutions and beneficial technical effects of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners. It should be understood that the specific implementation manners described in this specification are only for explaining the present invention and not for limiting the present invention.

[0039] As Figure 1 shown, the color three-dimensional imaging system based on three cameras includes two grayscale cameras and one color camera. Specifically, the two grayscale cameras are composed of a left camera and a right camera. Specifically, the left camera and the right camera are separated by a certain distance to form a binocular stereo vision system. The left camera and the right camera are used to collect grayscale images containing a target, and multiple fiducial points are set on the target. The color camera is used to collect a color image containing the target; the target is placed in the common effective field of view of the three cameras, so that the fiducial points on the target are within the effective field of view of the color three-dimensional imaging system, and the fiducial points can be recognized in the images of the target collected by the left camera, the right camera and the color camera.

[0040] Please refer to Figure 2 , an embodiment of the present invention provides a calibration method for a color three-dimensional imaging system based on three cameras, which specifically includes the following steps:

[0041] Step S201, establish a color three-dimensional imaging system including two grayscale cameras and one color camera, where the two grayscale cameras are composed of a left camera and a right camera.

[0042] In step S201, the target is placed at different angles and positions within the common effective area of the three cameras, so that the left camera, the right camera and the color camera respectively collect a first image, a second image and a third image containing the target at different positions.

[0043] In step S202 , the left camera, the right camera and the color camera are used to capture the first image, the second image and the third image containing the target, respectively, and obtain the pixel coordinates of the marker point in the first image, the second image and the third image, respectively.

[0044] In step S202, the left camera, the right camera, and the color camera respectively capture images corresponding to the target, and then the first image captured by the left camera, the second image captured by the right camera, and the third image captured by the color camera are respectively identified to obtain feature information in the image, thereby determining the image area corresponding to the marker point in the image. Specifically, a plurality of marker points are provided on the target, and the target is placed in the effective field of view of the system so that the left camera, the right camera, and the color camera respectively capture the first image, the second image, and the third image containing the target at different positions. In this embodiment, the marker point is preferably a circular pattern (in other embodiments, it can also be other patterns such as a triangle or a rectangle). The center position of the image area corresponding to the marker point is determined by locating the center of the circle, and the coordinates corresponding to the center position are obtained (the coordinates are the pixel coordinates of the pixel point corresponding to the center position in the image), which are the pixel coordinates corresponding to the marker point.

[0045] In step S203, the binocular system composed of the left and right cameras is calibrated according to the pixel coordinates of the marker points in the first image and the second image using the back projection principle of the left and right cameras to obtain first calibration parameters of the binocular system. The first calibration parameters include at least the extrinsic parameter matrix of the left camera, the transformation parameters of the left and right cameras, the internal parameters, and the distortion parameters.

[0046] Please also refer to Figure 3 , step S203 may include the following sub-steps:

[0047] Step S301 : establishing linear projection models of the left and right cameras using a back-projection distortion model according to the pixel coordinates of the marker points in the first image and the second image.

[0048] Specifically, the relationship between pixel coordinates and distorted image coordinates is calculated as the first transformation relationship, where the first transformation relationship includes the internal parameters of the left camera or the right camera. The binocular system consists of left and right cameras. The distorted image coordinates corresponding to the left camera are obtained based on the pixel coordinates of the marker point in the first image, and the distorted image coordinates corresponding to the right camera are obtained based on the pixel coordinates of the marker point in the second image. The first transformation relationship is calculated under the left camera, that is, the transformation relationship between the pixel coordinates of the marker point in the first image and the distorted image coordinates corresponding to the left camera is as follows:

[0049]

[0050] in, is the homogeneous representation of the distorted image coordinate x' corresponding to the left camera l , is the homogeneous representation of the pixel coordinate m' corresponding to the left camera l , and K l is the internal parameter of the left camera

[0051] Furthermore, calculate the first transformation relation under the right camera, that is, the transformation relation between the pixel coordinate of the fiducial point in the second image and the distorted image coordinate corresponding to the right camera is as follows:

[0052]

[0053] wherein is the homogeneous representation of the distorted image coordinate x' corresponding to the right camera r , is the homogeneous representation of the pixel coordinate m' corresponding to the right camera r , and K r is the internal parameter of the right camera

[0054] Specifically, calculate the relation for transforming from the distorted image coordinate to the undistorted image coordinate as the second transformation relation, where the second transformation relation includes the distortion parameters of the left camera or the right camera. In this embodiment, the binocular system establishes a linear model through the back-projection principle, which can transform the distorted image coordinate corresponding to the left camera into the undistorted image coordinate corresponding to the left camera, and transform the distorted image coordinate corresponding to the right camera into the undistorted image coordinate corresponding to the right camera. In the back-projection model, the distortion can be directly removed from the distorted image coordinate to obtain the image coordinate under the linear model, that is, the undistorted image coordinate. That is, the second transformation relation can be expressed as:

[0055] X = X' - Δ(X')

[0056]

[0057] wherein X is the undistorted image coordinate, X' is the distorted image coordinate, and Δ(X') is an explicit function of the image distortion X'; r 2 = x' 2 + y' 2 , and k1, k2, k3 are radial distortion parameters, and p1, p2, p3 are tangential distortion parameters

[0058] Specifically, substitute the pixel coordinates of the fiducial point in the first image and the second image into the above first transformation relation and second transformation relation respectively to obtain the linear projection models of the left and right cameras. In this embodiment, use the back-projection principle to substitute the distorted image coordinate x' corresponding to the left camera l and the distorted image coordinate x' corresponding to the right camerar , the undistorted image coordinates x corresponding to the left and right cameras can be obtained respectively l and x r .

[0059] Step S302: Obtain the observed value of the three-dimensional coordinates of the fiducial point by using the principle of ray intersection of triangulation based on the linear projection models of the left and right cameras.

[0060] In step S302, under the left camera, the transformation relationship between the camera coordinates in the camera coordinate system corresponding to the fiducial point and the actual value of the three-dimensional coordinates of the fiducial point is as follows:

[0061] X l = R l X w + T l

[0062] where, X l is the camera coordinate corresponding to the fiducial point in the left camera coordinate system, X w is the world coordinate corresponding to the fiducial point on the target, and R l , T l are the external parameter matrices of the left camera. Specifically, the world coordinate of the fiducial point in the world coordinate system is known, that is, the actual value of the three-dimensional coordinates.

[0063] Furthermore, through the undistorted image coordinates x l and x r corresponding to the left and right cameras obtained in step S301, the transformation relationship between the undistorted image coordinates corresponding to the fiducial point and the corresponding camera coordinates can be determined, that is:

[0064]

[0065]

[0066] where, are the homogeneous expressions of x l and x r respectively, is the homogeneous expression of X l , and R s , T s are the transformation parameters of the left and right cameras.

[0067] Furthermore, from the corresponding relationships obtained in step S301 and step S302, that is, the principle of ray intersection of triangulation, the observed value of the three-dimensional coordinates of the fiducial point can be obtained, which includes the first calibration parameter to be optimized and can be specifically expressed as:

[0068]

[0069] Step S303: Construct a first optimization objective function based on the Euclidean distance deviation between the observed value and the actual value of the three-dimensional coordinates of the fiducial points, and minimize the first optimization objective function through iterative optimization calculations to obtain the first calibration parameters.

[0070] In step S303, the left and right cameras respectively capture the fiducial points with j serial numbers at the i-th different positions of the target. The more fiducial points are obtained and the more different positions of the target are collected, that is, the more corresponding first images and second images there are, the more accurate the calibration result will be. Then, by minimizing the Euclidean distance deviation between the observed value and the known actual value of the three-dimensional coordinates of the fiducial points, the first optimization objective function is constructed.

[0071] Specifically, the first optimization objective function is

[0072] Using the Gauss-Newton method of the nonlinear least squares method to solve the above first optimization objective function, the data results of the first calibration parameters can be obtained, that is, at least including the external parameter matrix (R l , T l ) of the left camera, the transformation parameters (R s , T s ) between the left and right cameras, the internal parameters (K l , K r ) and the distortion parameters (d l , d r ), where the distortion parameters of the left and right cameras both include radial distortion parameters and tangential distortion parameters. That is to say, in the calibration scheme given in this embodiment, through the iterative optimization of the first optimization objective function, the calibration of the binocular system including the left and right cameras is completed.

[0073] Step S204: Calibrate the color camera according to the external parameter matrix of the left camera and the pixel coordinates of the fiducial points in the third image using the forward projection principle of the color camera to obtain the second calibration parameters. The second calibration parameters at least include the transformation parameters corresponding to the color camera and the left camera, the internal parameters of the color camera, and the distortion parameters.

[0074] Among them, please refer to Figure 4 together. Step S204 may include the following sub-steps:

[0075] Step S401: Transform the world coordinate system into the camera coordinate system corresponding to the left camera according to the external parameter matrix of the left camera, and establish the transformation relationship between the camera coordinate system corresponding to the left camera and the camera coordinate system corresponding to the color camera to obtain the expression of the three-dimensional coordinates of the fiducial points in the camera coordinate system corresponding to the color camera.

[0076] Specifically, according to the external parameter matrix of the left camera calibrated in step S203, the world coordinates of the fiducial points can be transformed into the camera coordinates corresponding to the left camera, that is:

[0077] X l = R l X w + T l

[0078] Furthermore, based on the camera coordinates corresponding to the left camera above, the transformation relationship between the camera coordinate system corresponding to the left camera and the camera coordinate system corresponding to the color camera can be determined, that is:

[0079] X c = R c X l + T c

[0080] where R c , T c are the transformation parameters corresponding to the color camera and the left camera, and X c is the camera coordinate corresponding to the fiducial point in the image acquired by the color camera.

[0081] Step S402: According to the expression of the three-dimensional coordinates of the fiducial points in the camera coordinate system corresponding to the color camera, use the internal parameters of the color camera and the distortion model of forward projection to obtain the theoretical pixel coordinates corresponding to the fiducial points in the projection image of the color camera.

[0082] In step S402, the color camera is calibrated using the forward projection model. First, according to the expression of the three-dimensional coordinates of the fiducial points in the camera coordinate system corresponding to the color camera, the normalized image coordinates of the fiducial points in the projection image of the color camera are obtained. These image coordinates are the undistorted image coordinates of the fiducial points on the imaging plane obtained by the forward projection of the color camera, as follows:

[0083] x n = [X c1 / X c3 ; X c2 / X c3

[0084] where x n is the undistorted image coordinate corresponding to the color camera, and [X c1 ] , X c2 , X c3 represents a three-dimensional coordinate.

[0085] ​Further, calculate the relational expression for transforming the normalized image coordinates of the fiducial points in the color camera's projected image to the distorted image coordinates as the third transformation relational expression, where the third transformation relational expression includes the distortion parameters of the color camera. In the forward projection model, considering that lens distortion can be obtained by directly adding the distortion to the undistorted image coordinates, that is, the third transformation relational expression can be expressed as:

[0086] X' c = X c + Δ(X c )

[0087]

[0088] where X' c is the distorted image coordinate, X c is the normalized image coordinate, and Δ(X c ) is an explicit function of the lens distortion of X c ; k1, k2, and k3 are radial distortion parameters, and p1, p2, and p3 are tangential distortion parameters. Specifically, substituting the undistorted image coordinates corresponding to the color camera into the forward projection model, the distorted image coordinates corresponding to the color camera can be obtained.

[0089] Further, calculate the relational expression for transforming the distorted image coordinates in the third transformation relational expression to the theoretical pixel coordinates as the fourth transformation relational expression, where the fourth transformation relational expression includes the internal parameters of the color camera. The above-mentioned fourth transformation relational expression can be expressed as follows:

[0090]

[0091] where, is the homogeneous expression of the distorted image coordinates corresponding to the color camera, is the target pixel coordinate corresponding to the color camera, and KK c is the internal parameter matrix of the color camera.

[0092] Further, obtain the calculation expression of the theoretical pixel coordinates according to the above-mentioned third transformation relational expression and fourth transformation relational expression, and use it as the theoretical value of the pixel coordinates.

[0093] Step S403, construct a second optimization objective function based on the Euclidean distance deviation between the theoretical value of the pixel coordinates of the fiducial points and the actual pixel coordinates of the corresponding fiducial points in the third image, and minimize the second optimization objective function through iterative optimization calculations to obtain the second calibration parameters.

[0094] In step S403, the color camera captures the fiducial points with j serial numbers at the i-th different positions of the target, and the actual pixel coordinates of the fiducial points in the third image are obtained through image processing. Then, the Euclidean distance deviation between the theoretical value of the pixel coordinates of the fiducial point and the actual pixel coordinates of the fiducial point in the image obtained by the color camera is minimized, thereby establishing a second optimization objective function, where the second optimization objective function includes the second calibration parameters to be optimized.

[0095] Specifically, the second optimization objective function is

[0096] The Gauss-Newton method using the nonlinear least squares method is used to solve the above second optimization objective function, and the data results of the second calibration parameters can be obtained, that is, at least including the transformation parameters (R c , T c ) corresponding to the color camera and the left camera, the internal parameters KK c of the color camera, and the distortion parameters, where the distortion parameters of the color camera include both radial distortion parameters and tangential distortion parameters. That is to say, in the calibration scheme given in this embodiment, through the iterative optimization of the second optimization objective function, the calibration of the color camera is completed.

[0097] In another specific embodiment of the present invention, a color three-dimensional imaging system based on three cameras is further provided. As Figure 5 shown, the system specifically includes two grayscale cameras (11, 12) and a color camera 13. The two grayscale cameras (11, 12) are composed of a left camera 11 and a right camera 12. The left camera 11, the right camera 12, and the color camera 13 are used to collect the first image, the second image, and the third image containing the target 14 respectively, and multiple fiducial points are arranged on the target 14. The above system further includes a memory 15 and a processor 16 connected to the left camera 11, the right camera 12, and the color camera 13. Specifically, the memory 15 is used to store a computer program that can run on the processor 16, and when the processor 16 executes the above computer program, the calibration method of the color three-dimensional imaging system based on three cameras in the embodiment of the present invention is implemented.

[0098] The beneficial effects of the color three-dimensional imaging system based on three cameras and its calibration method provided by the embodiments of the present invention are as follows: In the specific calibration process, the back-projection principle is first used to reconstruct the imaging model of the binocular system including the left and right cameras, which is beneficial to directly removing the distortion from the image and completing the calibration of the binocular system; then the forward-projection principle is used to reconstruct the imaging model of the color camera to complete the calibration of the color camera, making the calibration optimization objective function consistent with the color three-dimensional reconstruction process, which is beneficial to improving the calibration accuracy.

[0099] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0100] The above are only the preferred embodiments of the present invention, and are further detailed descriptions of the present invention in combination with specific preferred implementation manners. It cannot be determined that the specific implementation of the present invention is limited to these descriptions. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A calibration method for a color three-dimensional imaging system based on three cameras, characterized in that, The method includes the following steps: Establish a color three-dimensional imaging system including two grayscale cameras and one color camera, where the two grayscale cameras consist of a left camera and a right camera; Collect a first image, a second image, and a third image containing a target through the left camera, the right camera, and the color camera respectively, and obtain the pixel coordinates of the fiducial points in the first image, the second image, and the third image respectively; wherein a plurality of fiducial points are arranged on the target; Calibrate the binocular system composed of the left and right cameras according to the pixel coordinates of the fiducial points in the first image and the second image by using the back-projection principle of the left and right cameras, and obtain the first calibration parameters of the binocular system. The first calibration parameters at least include the external parameter matrix of the left camera, the transformation parameters between the left and right cameras, the internal parameters, and the distortion parameters; Calibrate the color camera according to the external parameter matrix of the left camera and the pixel coordinates of the fiducial points in the third image by using the forward-projection principle of the color camera, and obtain the second calibration parameters. The second calibration parameters at least include the transformation parameters corresponding to the color camera and the left camera, the internal parameters of the color camera, and the distortion parameters; The step of calibrating the binocular system composed of the left and right cameras according to the pixel coordinates of the fiducial points in the first image and the second image by using the back-projection principle of the left and right cameras, and obtaining the first calibration parameters of the binocular system includes: Establish a linear projection model of the left and right cameras according to the pixel coordinates of the fiducial points in the first image and the second image by using the distortion model of back-projection; Obtain the observed values of the three-dimensional coordinates of the fiducial points according to the linear projection model of the left and right cameras by using the ray intersection principle of triangulation; Construct a first optimization objective function according to the Euclidean distance deviation between the observed values and the actual values of the three-dimensional coordinates of the fiducial points, and minimize the first optimization objective function through iterative optimization calculations to obtain the first calibration parameters; The step of establishing a linear projection model of the left and right cameras according to the pixel coordinates of the fiducial points in the first image and the second image by using the distortion model of back-projection includes: Calculate the relational expression for transforming from pixel coordinates to distorted image coordinates as the first transformation relational expression, and calculate the relational expression for transforming from distorted image coordinates to undistorted image coordinates as the second transformation relational expression, where the first transformation relational expression contains the internal parameters of the left camera or the right camera, and the second transformation relational expression contains the distortion parameters of the left camera or the right camera; Substitute the pixel coordinates of the fiducial points in the first image and the second image into the first transformation relational expression and the second transformation relational expression respectively to obtain the linear projection models of the left and right cameras; The step of calibrating the color camera according to the external parameter matrix of the left camera and the pixel coordinates of the fiducial points in the third image by using the forward-projection principle of the color camera, and obtaining the second calibration parameters includes: Transform the world coordinate system into the camera coordinate system corresponding to the left camera according to the external parameter matrix of the left camera, and establish the transformation relationship between the camera coordinate system corresponding to the left camera and the camera coordinate system corresponding to the color camera to obtain the expression of the three-dimensional coordinates of the fiducial point in the camera coordinate system corresponding to the color camera; According to the expression of the three-dimensional coordinates of the fiducial point in the camera coordinate system corresponding to the color camera, use the internal parameters of the color camera and the distortion model of forward projection to obtain the theoretical pixel coordinate value corresponding to the fiducial point in the projection image of the color camera; Construct a second optimization objective function according to the Euclidean distance deviation between the theoretical pixel coordinate value of the fiducial point and the actual pixel coordinate of the corresponding fiducial point in the third image, and minimize the second optimization objective function through iterative optimization calculation to obtain the second calibration parameter; The step of using the internal parameters of the color camera and the distortion model of forward projection to obtain the theoretical pixel coordinate value corresponding to the fiducial point in the projection image of the color camera according to the expression of the three-dimensional coordinates of the fiducial point in the camera coordinate system corresponding to the color camera includes: Obtain the normalized image coordinates of the fiducial point in the projection image of the color camera according to the expression of the three-dimensional coordinates of the fiducial point in the camera coordinate system corresponding to the color camera; Calculate the relational expression for transforming the normalized image coordinates of the fiducial point in the projection image of the color camera into the distorted image coordinates as the third transformation relational expression, and the third transformation relational expression includes the distortion parameters of the color camera; Calculate the relational expression for transforming the distorted image coordinates in the third transformation relational expression into the theoretical pixel coordinates as the fourth transformation relational expression, and the fourth transformation relational expression includes the internal parameters of the color camera; Obtain the calculation expression of the theoretical pixel coordinates according to the third transformation relational expression and the fourth transformation relational expression and use it as the theoretical value of the pixel coordinates.

2. The calibration method of the color three-dimensional imaging system based on three cameras according to claim 1, characterized in that, The first optimization objective function includes the first calibration parameter to be optimized, and the distortion parameters of the left and right cameras respectively include the radial distortion parameter and tangential distortion parameter corresponding to the left camera, and the radial distortion parameter and tangential distortion parameter corresponding to the right camera.

3. The calibration method of the color three-dimensional imaging system based on three cameras according to claim 1, characterized in that The second optimization objective function includes the second calibration parameter to be optimized, and the distortion parameters of the color camera include the radial distortion parameter and tangential distortion parameter corresponding to the color camera.

4. The calibration method of the color three-dimensional imaging system based on three cameras according to claim 1, characterized in that, Place the target at different angles and positions within the common effective area of the three cameras, so that the left camera, the right camera and the color camera respectively collect the first image, the second image and the third image containing the target at different positions.

5. A color three-dimensional imaging system based on three cameras, comprising two grayscale cameras and one color camera. The two grayscale cameras consist of a left camera and a right camera. The left camera, the right camera, and the color camera are respectively used to acquire a first image, a second image, and a third image containing a target, and a plurality of fiducial points are arranged on the target; characterized in that, The system further includes a memory and a processor connected to the left camera, the right camera and the color camera; the memory is used to store a computer program that can run on the processor, and when the processor executes the computer program, it implements the calibration method of the color three-dimensional imaging system based on three cameras as described in any one of claims 1-4.

Images