Scale recovery method for performing three-dimensional reconstruction by using camera in vibration environment

Abstract

The invention relates to the technical field of photogrammetry, in particular to a scale recovery method for performing three-dimensional reconstruction by using a camera in a vibration environment, which comprises the following steps of: S1, setting a scene; s2, at each moment, respectively resolving a rotation matrix RML and a translation vector tML of the middle camera relative to the left calibration pattern, and a rotation matrix RMR and a translation vector tMR of the middle camera relative to the right calibration pattern; s3, resolving a relative pose between the left camera and the right camera at each moment; and S4, calculating the baseline length and determining a projection matrix. By adding the middle camera and the calibration plate, the length of the base line between the optical centers of the left camera and the right camera during each shooting is determined, and the problem that the three-dimensional data of each shot image during camera vibration can be recovered to the initial scale is solved.

Description

Technical Field

[0001] The present invention relates to photogrammetry technology, in particular to computer stereo vision three-dimensional reconstruction technology, and specifically to a scale recovery method for three-dimensional reconstruction using a camera in a vibration environment. Background Art

[0002] Performing three-dimensional reconstruction on each image captured simultaneously by two or more fixed cameras is a traditional topic. However, in a vibration environment, the cameras will shake irregularly, resulting in changes in the initial baseline between the cameras in a multi-camera system, affecting the scale of three-dimensional reconstruction. At the same time, since each scene captured by the cameras is changing, it is difficult to use scene information to recover the scale of three-dimensional reconstruction. Summary of the Invention

[0003] Aiming at the problems raised in the background art, the object of the present invention is to propose a scale recovery method for three-dimensional reconstruction using a camera in a vibration environment, which determines the baseline length of the binocular cameras at each shooting by adding an intermediate camera and a calibration board, and solves the problem that the three-dimensional data of each captured image can be restored to the initial scale.

[0004] To achieve the above object, the present invention adopts the following technical solutions: A scale recovery method for three-dimensional reconstruction using a camera in a vibration environment, comprising the following steps: Step S1, set the scene; in a vibration environment, deploy a left camera, a right camera, and an intermediate camera facing the observation scene. The left camera and the right camera observe the scene from the left and the right respectively, and the intermediate camera is deployed behind the left camera and the right camera. Since the cavity to be reconstructed deforms dynamically strongly, resulting in strong ground vibrations, the relative poses of the left and right cameras with respect to the intermediate camera change at each moment. The left camera is fixedly connected to the left calibration board, and the right camera is fixedly connected to the right calibration board. The relative pose of the left camera with respect to the left calibration board remains unchanged, and the relative pose of the right camera with respect to the right calibration board remains unchanged. All cameras work synchronously. At each shooting, the field of view of the intermediate camera includes the left camera, the right camera, the left calibration board, and the right calibration board; Step S2, at each moment, respectively solve the rotation matrix R ML and the translation vector t ML of the intermediate camera with respect to the left calibration board, and the rotation matrix R MR and the translation vector t MR of the intermediate camera with respect to the right calibration board; Step S3, at each moment, based on the solution results of step 2, using the left camera as the main camera, solve the relative pose between the left camera and the right camera; the relative pose between the left camera and the right camera includes the rotation matrix R and the translation vector t from the right camera to the left camera; Step S4: Based on the rotation matrix R and the translation vector t at each moment, calculate in real time the baseline length between the optical centers of the left camera and the right camera, and respectively determine the projection matrices of the left camera and the right camera.

[0005] In the said step S1, shock-absorbing platforms are installed on both the left camera, the right camera and the middle camera, and protective cases are equipped; both the left calibration board and the right calibration board are checkerboard calibration boards.

[0006] The said step S3 includes the following steps: S3.1: Set the rotation matrix of the left camera relative to the left calibration board as R L , and the translation vector as t L , then the rotation matrix from the left camera to the middle camera is R ML -1 R L , and the translation vector is -R ML -1 (t ML - t L ); Set the rotation matrix of the right camera relative to the right calibration board as R R , and the translation vectors are respectively t R , then the rotation matrix from the right camera to the middle camera is R MR -1 R R , and the translation vector is -R MR -1 (t MR - t R ); S3.2: Taking the middle camera as a medium, the rotation matrix R and the translation vector t from the right camera to the left camera can be obtained as follows:

[0007] Simplified to:

[0008] In the said step S4, the baseline length between the optical centers of the left camera and the right camera is the modulus of the translation vector t obtained in step S3.

[0009] In the said step S4, the projection matrices of the left camera and the right camera are obtained according to the following method: If the origin of the world coordinate system is constructed at the optical center of the left camera, and the projection matrix of the left camera at each moment is P L = K L [I | O], then the projection matrix P R of the right camera at each moment = K R [R | t], K L and K Rare the internal parameter matrices of the left and right cameras respectively, I is a 3×3 identity matrix, O is a 3×1 zero matrix, R is the rotation matrix from the right camera to the left camera, and t is the translation vector from the right camera to the left camera.

[0010] The present invention has the following beneficial effects: The present invention can solve the scale problem caused by the change of the baseline length at each moment in the case of camera shaking. According to the method of the present invention, the relative pose between every two cameras in a multi-camera system can be calculated, so that the scale of the three-dimensional reconstruction is kept consistent with the initial scale in real time, and the accuracy of the three-dimensional reconstruction of the stereo model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of the layout pose of the middle camera relative to the left camera and the right camera of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments.

[0013] In computer vision, three-dimensional reconstruction refers to the process of reconstructing three-dimensional information based on single-view or multi-view images. Since the information of a single video is incomplete, three-dimensional reconstruction requires the use of empirical knowledge. The three-dimensional reconstruction of multi-views (similar to human binocular positioning) is relatively easy. The method is to first calibrate the camera, that is, calculate the relationship between the image coordinate system of the camera and the world coordinate system, and then use the information in multiple two-dimensional images to reconstruct three-dimensional information. However, when the scene is constantly changing and in a vibrating environment, the changes in the relationship between the image coordinate systems of two or more cameras and the world coordinate system are inconsistent, resulting in continuous changes in the basic scale, and the reconstructed stereo images will be distorted and the accuracy will be reduced.

[0014] To solve the above problems, a method for scale recovery of three-dimensional reconstruction using cameras in a vibrating environment according to the present invention includes the following steps: Step S1, set the scene; in a vibrating environment, deploy a left camera, a right camera and a middle camera facing the observation scene. The left camera and the right camera observe the scene from the left and right sides respectively, and the middle camera is deployed behind the left camera and the right camera; since the cavity to be reconstructed deforms dynamically strongly, resulting in strong ground vibration, the relative poses of the left and right cameras relative to the middle camera change at each moment; the left camera is fixedly connected to the left calibration board, and the right camera is fixedly connected to the right calibration board. The relative pose of the left camera relative to the left calibration board remains unchanged, and the relative pose of the right camera relative to the right calibration board remains unchanged; all cameras work synchronously. Each time a shot is taken, the middle camera's field of view includes the left camera, the right camera, the left calibration board and the right calibration board; Step S2, at each moment, respectively solve the rotation matrix R of the middle camera relative to the left calibration board ML and the translation vector tML The rotation matrix R of the middle camera relative to the right calibration board MR and the translation vector t MR ; Step S3: At each moment, based on the calculation result of Step 2, calculate the relative pose between the left camera and the right camera; the relative pose between the left camera and the right camera includes the rotation matrix R and the translation vector t from the right camera to the left camera; Step S4: Based on the rotation matrix R and the translation vector t at each moment, calculate in real time the baseline length between the optical centers of the left camera and the right camera, and respectively determine the projection matrices of the left camera and the right camera.

[0015] In the said Step S1, shock-absorbing platforms are installed on the left camera, the right camera and the middle camera, and protective cases are equipped; both the left calibration board and the right calibration board are checkerboard calibration boards.

[0016] The said Step S3 includes the following steps: S3.1: Set the rotation matrix of the left camera relative to the left calibration board as R L and the translation vector as t L , then the rotation matrix from the left camera to the middle camera is R ML -1 R L and the translation vector is -R ML -1 (t ML -t L ); Set the rotation matrix of the right camera relative to the right calibration board as R R and the translation vectors are respectively t R , then the rotation matrix from the right camera to the middle camera is R MR -1 R R and the translation vector is -R MR -1 (t MR -t R ); S3.2: Using the middle camera as a medium, obtain the rotation matrix R and the translation vector t from the right camera to the left camera; as follows:

[0017] Simplified to:

[0018] In the said Step S4, the baseline length between the optical centers of the left camera and the right camera is the modulus of the translation vector t obtained in Step S3.

[0019] In step S4, the projection matrices of the left camera and the right camera are obtained as follows: If the origin of the world coordinate system is constructed at the optical center of the left camera, and the projection matrix of the left camera at each moment is P L = K L [I|O], then the projection matrix P R = K R [R|t], where K L and K R are the internal parameter matrices of the left and right cameras respectively, I is a 3×3 identity matrix, O is a 3×1 zero matrix, R is the rotation matrix from the right camera to the left camera, and t is the translation vector from the right camera to the left camera.

[0020] Both the left calibration board and the right calibration board are checkerboard calibration boards.

[0021] According to the above method, at each moment, the projection matrices of the left camera and the right camera in the world coordinate system can be determined, and thus the baseline length during 3D reconstruction at each moment can be determined. When the baseline length is the correct length, the distance between any two points in space can be correctly reconstructed, for example, d = |X1 - X2|. When the distance between any two points at each moment is determined, based on the fact that the scale during 3D reconstruction at each moment is always consistent with the true value, if the initial scale determined by the initial baseline is used as a reference, according to the change in the length of the baseline, the initial scale can be restored, so as to obtain the optimal image output.

[0022] Parts not detailed in the present invention are prior art.

Claims

1. A scale recovery method for three-dimensional reconstruction using a camera in a vibrating environment, characterized by: The following steps are involved: Step S1, setting the scene; In a vibrating environment, the left camera and the right camera are deployed facing the observation scene, and the left camera and the right camera observe the scene from the left and right sides respectively; the left camera is fixedly connected to the left calibration pattern, and the right camera is fixedly connected to the right calibration pattern. The middle camera is deployed behind the left camera and the right camera, and the relative position and pose of the left camera relative to the left calibration pattern remains unchanged, and the relative position and pose of the right camera relative to the right calibration pattern remains unchanged; all cameras work synchronously, and each time a shot is taken, the field of view of the middle camera includes the left camera, the right camera, the left calibration pattern, and the right calibration pattern; due to the strong dynamic deformation of the cavity to be reconstructed, the ground vibrates strongly, so the relative position and pose of the left and right cameras relative to the middle camera are changing at every moment; Step S2: At each moment, calculate the rotation matrix R of the middle camera relative to the left calibration pattern. ML and the translation vector t ML , the rotation matrix R of the middle camera relative to the right calibration pattern MR and the translation vector t MR ; Step S3: at each moment, based on the calculation result of step 2, calculate the relative position and posture between the left camera and the right camera; the relative position and posture between the left camera and the right camera includes the rotation matrix R and the translation vector t from the right camera to the left camera; Step S4: Based on the rotation matrix R and the translation vector t at each moment, the baseline length between the optical center of the left camera and the optical center of the right camera is calculated in real time, and the projection matrices of the left camera and the right camera are determined respectively.

2. The scale restoration method for three-dimensional reconstruction using a camera in a vibration environment according to claim 1, characterized in that: In step S1, the left camera, the right camera and the middle camera are all equipped with a shock-absorbing platform and a protective shell.

3. The scale restoration method for three-dimensional reconstruction using a camera in a vibration environment according to claim 1, characterized in that: The step S3 comprises the following steps: S3.

1. Set the rotation matrix of the left camera relative to the left calibration pattern to R L , the translation vector is t L , then the rotation matrix from the left camera to the middle camera is R ML -1 R L , the translation vector is -R ML -1 (t ML -t L ); Set the rotation matrix of the right camera relative to the right calibration pattern to R R , the translation vectors are t R , then the rotation matrix from the right camera to the middle camera is R MR -1 R R , the translation vector is -R MR -1 (t MR -t R ); S3.2, using the middle camera as the medium, we can get the rotation matrix R and translation vector t from the right camera to the left camera as follows: ， Simplified to: , 4. The scale restoration method for three-dimensional reconstruction using a camera in a vibration environment according to claim 1, characterized in that: In the step S4, the baseline length between the optical center of the left camera and the optical center of the right camera is the modulus length of the translation vector t obtained in the step S3.

5. The scale restoration method for three-dimensional reconstruction using a camera in a vibration environment according to claim 1, characterized in that: In step S4, the projection matrices of the left camera and the right camera are obtained according to the following method: If the origin of the world coordinate system is constructed at the optical center of the left camera, let the projection matrix of the left camera at each moment be P L = K L [I | O], then the projection matrix P of the right camera at each moment R = K R [R|t],K L and K R They are the intrinsic parameter matrices of the left and right cameras respectively, I is the 3×3 identity matrix, O is the 3×1 zero matrix, R is the rotation matrix from the right camera to the left camera, and t is the translation vector from the right camera to the left camera.

Images