Three-dimensional reconstruction method based on light field scanner

Abstract

The invention discloses a three-dimensional reconstruction method based on a light field scanner, and the method comprises the following steps: S1, carrying out the image collection of a collection object through a light field scanner, and obtaining a plurality of object images; s2, extracting local feature points, which can be repeatedly detected, on the object image obtained in the S1, and performing image capture on the object at a plurality of visual angles through a light field scanner, thereby providing rich visual angles and detail information for three-dimensional reconstruction; the average value of the parameter list, the average value of the rotation quaternion and the average value of the translation vector of the camera are obtained through multiple repeated experiments, and the internal parameter file and the external parameter file of each camera can be calculated according to the average values, so that the camera parameters do not need to be calculated again in the subsequent reconstruction operation, and the calculation amount in the reconstruction process is reduced; the whole reconstruction process is engineered, the number of manual clicks is reduced, the reconstruction speed is improved, and the technical problems that in the prior art, an image-based reconstruction method is long in time and high in cost are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and relates to a three-dimensional reconstruction method, in particular to a three-dimensional reconstruction method based on a light field scanner. Background Art

[0002] 3D reconstruction is of great significance in the preservation of cultural relics, particularly in digital archiving, restoration, research, and display. Cultural relics can be damaged or even lost due to natural aging, vandalism, or natural disasters. 3D reconstruction can accurately record the geometry, texture, and details of cultural relics, creating a digital archive for permanent preservation. This can also help restoration experts analyze damage and develop restoration plans. Furthermore, 3D reconstructed cultural relic models can be used in virtual museums, augmented reality (AR), and virtual reality (VR) displays, allowing more people to appreciate cultural relics without having to visit them in person.

[0003] Traditional 3D reconstruction methods (such as laser scanning) may take a long time and be costly. Rapid reconstruction technology (such as image-based reconstruction) can significantly shorten the reconstruction time and reduce manpower and material costs. In some emergency situations, such as natural disasters, wars, floods, and earthquakes, cultural relics may face serious threats. Rapid 3D reconstruction technology can complete the digital recording of cultural relics in a short time and at a low cost. However, existing traditional image-based reconstruction methods require one or more cameras. When there are multiple cameras and their relative positions are fixed, the positional relationship information between them is not utilized.

[0004] In summary, although image-based reconstruction can significantly shorten reconstruction time and reduce costs, there is still room for improvement under certain conditions. Specifically, the relative position relationship between cameras can be used to further optimize the reconstruction process and reduce the amount of computation. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a three-dimensional reconstruction method based on a light field scanner to solve the technical problems of the image-based reconstruction method in the existing technology that is still time-consuming and costly.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A three-dimensional reconstruction method based on a light field scanner comprises the following steps:

[0008] S1, using a light field scanner to collect images of the collection object to obtain multiple object images;

[0009] The light field scanner includes a circular base, on which a plurality of cameras are arranged, wherein camera number one is located at the center of the circular base, and the remaining cameras are evenly distributed around the periphery of the circular base, and i represents the serial number of the remaining cameras;

[0010] S2, extracting local feature points that can be repeatedly detected on the object image obtained in S1, matching similar local feature points between different object images, and establishing corresponding relationships, obtaining the camera pose based on the matched local feature points, and then obtaining the camera internal parameter file and the camera external parameter file;

[0011] The camera internal parameter file includes a camera model, a parameter list, and the width and height of the image;

[0012] The camera external parameter file includes the camera's rotation quaternion and translation vector;

[0013] S3, select a new acquisition object, increase the number of loops by 1, and return to S1 until the number of loops equals J, to obtain multiple camera internal parameter files and camera external parameter files;

[0014] S4, calculating the average value of all parameter lists in the multiple camera internal parameter files, and using the average value to replace the parameter list in each camera internal parameter file, thereby obtaining a new camera internal parameter file; and respectively calculating the average value of the rotation quaternion and the average value of the translation vector in the multiple camera external parameter files;

[0015] S5, select a set of rotation quaternions and translation vectors of camera number one; multiply them with the average value of the rotation quaternions and the average value of the translation vectors obtained in S4, to obtain the rotation quaternion and translation vector of each camera, and obtain a new camera extrinsic parameter file;

[0016] S6, converts the new camera intrinsic parameter file and the new camera extrinsic parameter file into standard format input files; generates a dense point cloud based on the new camera intrinsic parameter file and the new camera extrinsic parameter file, converts the dense point cloud into a mesh model through the Poisson reconstruction method, and projects the color of the object image obtained in S1 onto the surface of the mesh model to generate a texture map, thereby obtaining a 3D reconstruction result.

[0017] The present invention also includes the following technical features:

[0018] In step S4, the average value of the rotation quaternions in the plurality of camera extrinsic parameter files is calculated, which specifically includes the following steps:

[0019] Q1, calculate the rotation quaternion of camera 1 and each of the remaining cameras

[0020]

[0021] is the rotation quaternion between camera 1 and camera i in the j-th cycle;

[0022] QW i j is the real part of the corresponding rotation quaternion;

[0023] QY i j and is the imaginary part of the corresponding rotation quaternion;

[0024] is the rotation quaternion of camera No. 1 in the j-th cycle;

[0025] is the rotation quaternion of camera i in the j-th cycle;

[0026] Q2, j = j + 1, return to Q1, until j>J, output J × 14 rotation quaternions, J is the number of loops;

[0027] Q3, use the logarithmic-exponential mapping method to process J×14 rotation quaternions and obtain the average value of the rotation quaternions.

[0028] In step S4, the average value of the translation vectors in the plurality of camera extrinsic parameter files is calculated, which specifically includes the following steps:

[0029] P1, calculate the translation vector of camera 1 and each of the remaining cameras using the following formula

[0030]

[0031] in:

[0032] is the translation vector between camera 1 and camera i in the j-th cycle;

[0033] is the translation vector of camera No. 1 in the j-th cycle;

[0034] is the translation vector of camera i in the j-th cycle;

[0035] P2, j = j + 1, returns to P1, until j>J, output J × 14 translation vectors;

[0036] P3, calculate the mean of J×14 translation vectors to obtain the average value of the translation vectors.

[0037] Compared with the prior art, the present invention has the following beneficial technical effects:

[0038] The present invention uses a light field scanner to capture images of objects at multiple perspectives, providing rich perspectives and detailed information for three-dimensional reconstruction; through repeated experiments, the average value of the camera parameter list, the average value of the rotation quaternion, and the average value of the translation vector are obtained, and then the intrinsic parameter file and extrinsic parameter file of each camera can be calculated based on the above average values, so that subsequent reconstruction operations do not need to calculate the camera parameters, reducing the amount of calculation in the reconstruction process; the entire reconstruction process is engineered, reducing the number of manual clicks, and improving the reconstruction speed, solving the technical problems of the image-based reconstruction methods in the existing technology that are still time-consuming and costly. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Flow chart of the method of the present invention.

[0040] The specific contents of the present invention are further explained in detail below with reference to the embodiments. DETAILED DESCRIPTION

[0041] It should be noted that, unless otherwise specified, all components in the present invention are components known in the art.

[0042] Specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent modifications made on the basis of the technical solution of this application fall within the protection scope of the present invention.

[0043] The present invention provides a three-dimensional reconstruction method based on a light field scanner, comprising the following steps:

[0044] S1, using a light field scanner to collect images of the collection object to obtain multiple object images;

[0045] The light field scanner includes a circular base on which a plurality of cameras are arranged. Camera 1 is located at the center of the circular base, and the remaining cameras are evenly distributed around the periphery of the circular base. "i" represents the serial number of the remaining cameras.

[0046] S2, extracting local feature points that can be repeatedly detected on the object image obtained in S1, matching similar local feature points between different object images, and establishing corresponding relationships, obtaining the camera pose based on the matched local feature points, and then obtaining the camera internal parameter file and the camera external parameter file;

[0047] The camera internal parameter file includes the camera model, parameter list, and image width and height;

[0048] The camera external parameter file includes the camera's rotation quaternion and translation vector;

[0049] S3, select a new acquisition object, increase the number of loops by 1, and return to S1 until the number of loops equals J, to obtain multiple camera internal parameter files and camera external parameter files;

[0050] S4, calculating the average value of all parameter lists in the multiple camera internal parameter files, and using the average value to replace the parameter list in each camera internal parameter file, thereby obtaining a new camera internal parameter file; and respectively calculating the average value of the rotation quaternion and the average value of the translation vector in the multiple camera external parameter files;

[0051] S5, select a set of rotation quaternions and translation vectors of camera number one; multiply them with the average value of the rotation quaternions and the average value of the translation vectors obtained in S4, to obtain the rotation quaternion and translation vector of each camera, and obtain a new camera extrinsic parameter file;

[0052] S6, converts the new camera intrinsic parameter file and the new camera extrinsic parameter file into standard format input files; generates a dense point cloud based on the new camera intrinsic parameter file and the new camera extrinsic parameter file, converts the dense point cloud into a mesh model through the Poisson reconstruction method, and projects the color of the object image obtained in S1 onto the surface of the mesh model to generate a texture map, thereby obtaining a 3D reconstruction result.

[0053] In the above technical solution, images of objects are captured at multiple perspectives by a light field scanner, providing rich perspectives and detail information for three-dimensional reconstruction; the average value of the camera parameter list, the average value of the rotation quaternion, and the average value of the translation vector are obtained through repeated experiments, and then the intrinsic parameter file and extrinsic parameter file of each camera can be calculated based on the above average values, so that subsequent reconstruction operations do not need to calculate the camera parameters, reducing the amount of calculation in the reconstruction process; the entire reconstruction process is engineered to reduce the number of manual clicks and increase the speed of reconstruction, solving the technical problems of long time and high cost in the image-based reconstruction method in the existing technology.

[0054] The light field scanner is equipped with a capture controller, which connects to the fifteen cameras via a set of precision cables. These cables transmit control signals, ensuring precise synchronization of the cameras on each development board. The user points the light field scanner at an object and presses the capture controller to synchronize all fifteen cameras.

[0055] In step S4, the average value of the rotation quaternions in the plurality of camera extrinsic parameter files is calculated, which specifically includes the following steps:

[0056] Q1, calculate the rotation quaternion of camera 1 and each of the remaining cameras

[0057]

[0058] is the rotation quaternion between camera 1 and camera i in the j-th cycle;

[0059] QW i j is the real part of the corresponding rotation quaternion;

[0060] QY i j and is the imaginary part of the corresponding rotation quaternion;

[0061] is the rotation quaternion of camera No. 1 in the j-th cycle;

[0062] is the rotation quaternion of camera i in the j-th cycle;

[0063] Q2, j = j + 1, return to Q1, until j>J, output J × 14 rotation quaternions, J is the number of loops;

[0064] Q3, use the logarithmic-exponential mapping method to process J×14 rotation quaternions and obtain the average value of the rotation quaternions.

[0065] In the above scheme, the world coordinate system's origin is randomly chosen each time COLMAP performs SFM, so the extrinsic parameters obtained for the same camera in different experiments will vary. However, since the relative positions of the fifteen cameras in the light field scanner are fixed, the relative extrinsic parameters of the fifteen cameras are theoretically fixed. Therefore, for the camera extrinsic parameters, the average value is the relative rotation quaternion and relative translation vector between the two cameras.

[0066] In step S4, the average value of the translation vectors in the plurality of camera extrinsic parameter files is calculated, which specifically includes the following steps:

[0067] P1, calculate the translation vector of camera 1 and each of the remaining cameras using the following formula

[0068]

[0069] in:

[0070] is the translation vector between camera 1 and camera i in the j-th cycle;

[0071] is the translation vector of camera No. 1 in the j-th cycle;

[0072] is the translation vector of camera i in the j-th cycle;

[0073] P2, j = j + 1, returns to P1, until j>J, output J × 14 translation vectors;

[0074] P3, calculate the mean of J×14 translation vectors to obtain the average value of the translation vectors.

Claims

1. A three-dimensional reconstruction method based on a light field scanner, characterized in that: The steps include: S1, using a light field scanner to collect images of the collection object to obtain multiple object images; The light field scanner includes a circular base, on which a plurality of cameras are arranged, wherein camera number one is located at the center of the circular base, and the remaining cameras are evenly distributed around the periphery of the circular base, and i represents the serial number of the remaining cameras; S2, extracting local feature points that can be repeatedly detected on the object image obtained in S1, matching similar local feature points between different object images, and establishing corresponding relationships, obtaining the camera pose based on the matched local feature points, and then obtaining the camera internal parameter file and the camera external parameter file; The camera internal parameter file includes a camera model, a parameter list, and the width and height of the image; The camera external parameter file includes the camera's rotation quaternion and translation vector; S3, select a new acquisition object, increase the number of loops by 1, and return to S1 until the number of loops equals J, to obtain multiple camera internal parameter files and camera external parameter files; S4, calculating the average value of all parameter lists in the multiple camera internal parameter files, and using the average value to replace the parameter list in each camera internal parameter file, thereby obtaining a new camera internal parameter file; and respectively calculating the average value of the rotation quaternion and the average value of the translation vector in the multiple camera external parameter files; S5, select a set of rotation quaternions and translation vectors of camera number one; multiply them with the average value of the rotation quaternions and the average value of the translation vectors obtained in S4, to obtain the rotation quaternion and translation vector of each camera, and obtain a new camera extrinsic parameter file; S6, converts the new camera intrinsic parameter file and the new camera extrinsic parameter file into standard format input files; generates a dense point cloud based on the new camera intrinsic parameter file and the new camera extrinsic parameter file, converts the dense point cloud into a mesh model through the Poisson reconstruction method, and projects the color of the object image obtained in S1 onto the surface of the mesh model to generate a texture map, thereby obtaining a 3D reconstruction result.

2. The 3D reconstruction method based on a light field scanner according to claim 1, wherein: In step S4, the average value of the rotation quaternions in the plurality of camera extrinsic parameter files is calculated, which specifically includes the following steps: Q1, calculate the rotation quaternion of camera 1 and each of the remaining cameras is the rotation quaternion between camera 1 and camera i in the j-th cycle; QW i j is the real part of the corresponding rotation quaternion; QY i j and is the imaginary part of the corresponding rotation quaternion; is the rotation quaternion of camera No. 1 in the j-th cycle; is the rotation quaternion of camera i in the j-th cycle; Q2, j = j + 1, return to Q1, until j>J, output J × 14 rotation quaternions, J is the number of loops; Q3, use the logarithmic-exponential mapping method to process J×14 rotation quaternions and obtain the average value of the rotation quaternions.

3. The 3D reconstruction method based on a light field scanner according to claim 1, wherein: In step S4, the average value of the translation vectors in the plurality of camera extrinsic parameter files is calculated, which specifically includes the following steps: P1, calculate the translation vector of camera 1 and each of the remaining cameras using the following formula in: is the translation vector between camera 1 and camera i in the j-th cycle; is the translation vector of camera No. 1 in the j-th cycle; is the translation vector of camera i in the j-th cycle; P2, j = j + 1, returns to P1, until j>J, output J × 14 translation vectors; P3, calculate the mean of J×14 translation vectors to obtain the average value of the translation vectors.

Images