A three-dimensional reconstruction method and system based on an FPGA hardware accelerator

Abstract

The application discloses a three-dimensional reconstruction method and system based on an FPGA hardware accelerator, and the method comprises the following steps: acquiring image observation data and performing preprocessing to obtain preprocessed data; performing rotation optimization on the preprocessed data based on the FPGA hardware accelerator to estimate a global rotation matrix; and completing three-dimensional reconstruction according to the global rotation matrix. The system comprises a data preprocessing module, a rotation optimization module and a reconstruction module. By using the application, the rotation parameters of a camera in three-dimensional reconstruction can be optimized, and the operation efficiency can be improved and the power consumption can be reduced. The application can be widely applied to the field of three-dimensional reconstruction as a three-dimensional reconstruction method and system based on an FPGA hardware accelerator.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of three-dimensional reconstruction, in particular to a three-dimensional reconstruction method and system based on FPGA hardware accelerator. BACKGROUND

[0002] Three-dimensional reconstruction technology is to obtain two-dimensional data of a three-dimensional object in the real world by using a camera or other measurement methods, and then to restore the three-dimensional features of the object and reconstruct the three-dimensional state of the object based on the two-dimensional data by using image processing technology. With the rapid development of computer vision, information science and medical field, the three-dimensional reconstruction technology is used more and more widely in the fields of driverless, virtual reality and medical treatment and diagnosis of diseases, and gradually becomes a research hotspot.

[0003] The most important content in the three-dimensional reconstruction technology is the structure from motion (SFM). The result accuracy of global rotation estimation in the implementation process will affect the result quality of the subsequent steps. However, the relative pose estimation is limited to two views on one hand, and the rotation average does not directly use the measurement results of the image on the other hand, which limits the optimal implementation process of rotation estimation. In addition, the bundle adjustment method depends on parallel processing or distributed computing, and the commonly used software algorithm implementation process uses high power consumption to obtain high performance, which may bring very large power consumption and delay, which becomes the bottleneck of optimization implementation. SUMMARY

[0004] In order to solve the above technical problems, the purpose of the present application is to provide a three-dimensional reconstruction method and system based on FPGA hardware accelerator, which can optimize the rotation parameters of the camera in three-dimensional reconstruction, and improve the operation efficiency and reduce the power consumption.

[0005] The first technical solution adopted by the present application is: a three-dimensional reconstruction method based on FPGA hardware accelerator, comprising the following steps:

[0006] S1, obtaining image observation data and pre-processing to obtain pre-processed data;

[0007] S2, based on the FPGA hardware accelerator, the rotation optimization is performed on the pre-processed data, and the global rotation matrix is estimated;

[0008] S3, completing three-dimensional reconstruction according to the global rotation matrix.

[0009] Further, the step of obtaining image observation data and pre-processing to obtain pre-processed data specifically comprises:

[0010] S11, obtaining image observation data and classifying and converting the image observation data to obtain various types of data;

[0011] S12, allocate data memory to each type of data and store to the on-chip dual-port RAM of the FPGA hardware accelerator;

[0012] S13, complete the calculation of the cost function irrelevant to the rotation parameters, including the sum of the projection products of the normalized coordinates of the points commonly observed by each two cameras on any two coordinate axes;

[0013] S14, obtain the preprocessed data.

[0014] Further, the step of estimating the global rotation matrix based on the FPGA hardware accelerator and the rotation optimization of the preprocessed data specifically includes:

[0015] S21, convert the Rodrigues vector in the preprocessed data into a rotation matrix;

[0016] S22, calculate the minimum eigenvalue based on the preprocessed data and the rotation matrix to obtain the cost function value;

[0017] S23, calculate the Jacobian matrix based on the analytical differential calculation method;

[0018] S24, construct the parameter equation according to the Jacobian matrix and solve based on the matrix inversion method to obtain the change amount of the Rodrigues rotation vector;

[0019] S25, judge whether the cost function value meets the iteration condition, receive the change amount of the Rodrigues rotation vector and return to step S21, judge whether the cost function value does not meet the iteration condition, and update the trust region related parameters;

[0020] S26, loop steps S21-S25 until it is judged that the iteration condition is exceeded, and the rotation matrix calculated by the current iteration number is the global rotation matrix.

[0021] Further, the calculation formula of the rotation matrix is as follows:

[0022]

[0023] In the above formula, R represents the rotation matrix, r=[r1,r2,r3] T represents the Rodrigues vector, · T represents the transpose of the rotation vector, θ represents the rotation angle, r1, r2, r3 represent the projections of the rotation vector on the mutually orthogonal x-axis, y-axis and z-axis respectively.

[0024] Further, the step of calculating the minimum eigenvalue based on the preprocessed data and the rotation matrix to obtain the cost function value specifically includes:

[0025] Based on the preprocessed data, a symmetric matrix is calculated in combination with the rotation matrix;

[0026] According to the elements in the symmetric matrix, the remaining correlation functions are calculated;

[0027] According to the remaining correlation functions, the eigenvalues are solved;

[0028] The eigenvalues are cumulatively summed to obtain the cost function value.

[0029] Further, the calculation formula of the symmetric matrix is as follows:

[0030]

[0031] In the above formula, n represents the number of points commonly observed between each two cameras, i represents the i th point, f i represents the normalized camera coordinates of the point in one of the cameras, R represents a rotation matrix, f i represents the normalized camera coordinates of the point in the other camera, · T represents the transpose of the matrix.

[0032] Further, the calculation formula of the Jacobian matrix is as follows:

[0033]

[0034] In the above formula, f x and f y represent the camera focal lengths obtained by the linear perspective law, X', Y' and Z' represent the three-dimensional coordinates of the three-dimensional point in the camera coordinate system.

[0035] Further, the parameter equation is represented as follows:

[0036] (J T J+μI)Δp=g

[0037] In the above formula, J represents the Jacobian matrix, μ represents the trust region radius, I represents the unit matrix, Δp represents the change amount of the rotation vector, and g represents the gradient.

[0038] Further, the matrix inversion method specifically comprises:

[0039] According to the parameter equation, the adjoint matrix is calculated;

[0040] According to the elements in the adjoint matrix, the determinant of the block matrix is calculated and the reciprocal is obtained, to obtain the reciprocal of the determinant of the block matrix;

[0041] The adjoint matrix and the reciprocal of the determinant of the block matrix are multiplied to obtain the corresponding inverse matrix.

[0042] The second technical solution adopted by the present application is a three-dimensional reconstruction system based on an FPGA hardware accelerator, comprising:

[0043] A data preprocessing module is configured to acquire image observation data and perform preprocessing to obtain preprocessed data.

[0044] A rotation optimization module is configured to perform rotation optimization on the preprocessed data based on an FPGA hardware accelerator to estimate a global rotation matrix.

[0045] A reconstruction module is configured to complete three-dimensional reconstruction according to the global rotation matrix.

[0046] The method and system have the advantages that the FPGA-based parallel computation is used to further optimize the result of the global rotation estimation after three-dimensional reconstruction, thereby improving the accuracy, and the method has lower power consumption and faster operation speed than a pure software implementation. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a step flowchart of a three-dimensional reconstruction method based on an FPGA hardware accelerator according to the present application;

[0048] Figure 2 is a structural schematic diagram of rotation matrix calculation according to a specific embodiment of the present application;

[0049] Figure 3 is a schematic diagram of double-view epipolar constraint according to a specific embodiment of the present application;

[0050] Figure 4 is a flowchart of architecture implementation of cost function calculation according to a specific embodiment of the present application;

[0051] Figure 5 is a structural schematic diagram of matrix inversion in parameter equation solution according to a specific embodiment of the present application;

[0052] Figure 6 is an iteration schematic diagram of rotation optimization according to a specific embodiment of the present application;

[0053] Figure 7 is a relative error comparison diagram of the rotation optimization method and the global rotation estimation value after rotation averaging according to the present application;

[0054] Figure 8 is a structural block diagram of a three-dimensional reconstruction system based on an FPGA hardware accelerator according to the present application. DETAILED DESCRIPTION

[0055] The present application will be further described in detail below in combination with the drawings and specific embodiments. The step numbers in the following embodiments are only set for the purpose of facilitating description and explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0056] As Figure 1As shown, the application provides a three-dimensional reconstruction method based on FPGA hardware accelerator, which comprises the following steps:

[0057] S1, obtaining image observation data and pre-processing to obtain pre-processed data;

[0058] S11, obtaining image observation data and classifying and converting the image observation data to obtain various types of data;

[0059] Specifically, the parameter types include but are not limited to camera rotation vector, camera translation parameter, camera external parameter, observation coordinate and three-dimensional point world coordinate. The floating-point parameters of the preliminary pose estimation of the camera are converted into fixed-point numbers for easy FPGA operation, because the FPGA can only process fixed-point numbers and cannot directly process floating-point numbers.

[0060] S12, assigning data memory to each type of data and storing it to the on-chip dual-port RAM of the FPGA hardware accelerator to obtain pre-processed data.

[0061] Specifically, the data memory is reasonably allocated according to the numerical size and the characteristics of signed and unsigned numbers of each part of the data, which is beneficial to reduce the error influence caused by precision loss, and the converted data is uniformly stored in the on-chip dual-port RAM of the FPGA for subsequent calculation;

[0062] S13, completing the calculation of the cost function irrelevant to the rotation parameter, including the sum of the projection products of the normalized coordinates of the points commonly observed by each two cameras on any two coordinate axes;

[0063] Specific calculation And Obtain pre-processed data.

[0064] In the above formula, n represents the number of points commonly observed between each two cameras, i represents the i-th point, f i = [f xi ,f yi ,f zi ] T represents the normalized camera coordinates of the point in one of the cameras, f' i represents the normalized camera coordinates of the point in the other camera, T represents the transpose of the matrix, F' i represents f' i And the point multiplication result of

[0065] S14, obtaining pre-processed data.

[0066] S2, based on the FPGA hardware accelerator, performing rotation optimization on the pre-processed data to estimate the global rotation matrix;

[0067] S21. Convert the Rodriguez vector in the preprocessed data into a rotation matrix;

[0068] S211. Calculate the relevant values ​​of θ and store them in RAM, such as... Figure 2 The diagram shows the hardware structure of the rotation matrix construction module, which aims to implement the Rodriguez rotation formula and transform the rotation vector r into a rotation matrix R. First, three multipliers are used to calculate r1 in parallel. 2 r2 2 and r3 2 Next, the square root is calculated using the successive approximation algorithm, and the corresponding reciprocal is calculated using the shift and subtraction method. The corresponding trigonometric function values ​​sinθ and cosθ are then found using a pre-calculated lookup table.

[0069] S212. Convert the Rodriguez vector representing the camera parameters into a rotation matrix, as shown in the following formula:

[0070]

[0071]

[0072] In the above formula, R represents the rotation matrix, and r = [r1, r2, r3]. T Denotes the Rodriguez vector, · T Let r1, r2, and r3 represent the transpose of the rotation vector, θ represent the rotation angle, and r1, r2, and r3 represent the projections of the rotation vector onto the mutually orthogonal x-axis, y-axis, and z-axis, respectively.

[0073] S22. Based on the preprocessed data, the minimum eigenvalue is calculated using the rotation matrix to obtain the cost function value;

[0074] Specifically, by Figure 3 The schematic diagram of the epipolar constraint in the dual-view diagram can be used to derive the epipolar error in rotation optimization. Then, combining the least squares approach, the cost function of the accelerator is obtained as follows:

[0075]

[0076] The calculation process of the cost function is as follows: Figure 4 .

[0077] S221. Calculate the symmetric matrix based on the preprocessed data and the rotation matrix.

[0078] Specifically, we compute all elements of a symmetric matrix M, which has a dimension of 3×3. Considering that matrix M is a symmetric matrix, we only need to compute the 6 elements in the upper triangular region, thus saving 33% of resources. The expression for matrix M is:

[0079]

[0080] In the above formula, n represents the number of points commonly observed between every two cameras, i represents the i-th point, f i represents the normalized camera coordinates of the point in one of the cameras, R represents a rotation matrix, f i represents the normalized camera coordinates of the point in another camera, and · T represents the transpose of a matrix.

[0081] According to the operation rule of vector cross product, r i is defined as the row vector of matrix R, m ij represents the element in matrix M, and the calculation formula of each element is as follows:

[0082]

[0083]

[0084]

[0085]

[0086]

[0087]

[0088] S222, calculating the remaining correlation functions according to the elements in the symmetric matrix;

[0089] b1 = -m 11 -m 22 -m 33 ,

[0090]

[0091]

[0092]

[0093]

[0094]

[0095] In the above formula, m 11 corresponds to the element in the first row and the first column of matrix M, and the remaining data is the same.

[0096] S223, solving the eigenvalue according to the remaining correlation functions;

[0097] Specifically, the formula is as follows:

[0098] λ M = (-b1-2k) / 3

[0099] S224, accumulate the minimum eigenvalue to obtain a cost function value.

[0100] Specifically, the minimum eigenvalue λ is obtained and accumulated to obtain a cost function value:

[0101]

[0102] Where ε represents a set of all camera pairs, and (j, k) represents a corresponding set of camera pairs.

[0103] S23, calculate the Jacobian matrix based on the analytical differential calculation method;

[0104] Specifically, the calculation formula is:

[0105]

[0106] In the above formula, f x and f y are the focal lengths of the cameras obtained by the linear perspective law, and X', Y' and Z' represent the three-dimensional coordinates of the three-dimensional point in the camera coordinate system.

[0107] The product of the Jacobian matrix J T J is used to approximate the Hessian matrix, and in the Jacobian matrix calculation unit, since the partial derivatives of each set of camera pairs are calculated without interference, the Jacobian matrix calculation unit can be duplicated to reduce the Jacobian matrix calculation time and improve the parallelism of the accelerator calculation. If it is duplicated into 5 Jacobian matrix calculation units, the Jacobian matrix calculation time will be reduced by about 80%.

[0108] S24, solve the parameter equation based on the matrix inversion method to obtain the change amount of the Rodrigues rotation vector;

[0109] In the existing technical implementation, the Cholesky decomposition is used to solve the equation (J T J+μI)Δp=g, which contains square root, division and other calculation processes, especially multiple square root operations are interdependent and cannot be parallelized. In addition, the Cholesky decomposition requires a software and hardware collaborative design, that is, the algorithm module of the Cholesky decomposition process is implemented by the PS end software, and after the result is obtained, the data is transmitted back to the hardware logic circuit in the PL end by using the direct memory access unit DMA transmission mode based on the AXI-4 protocol. However, the DMA transmission inevitably causes the inconsistency between the Cache data and the memory data.

[0110] In order to solve this problem, for the solution equation stage, considering (J TThe matrix composed of J+μI is reversible, and is characterized by a block diagonal matrix, each diagonal block of which is a three-order symmetric matrix, so that the matrix inversion is used to realize the solution of the equation. Based on the linear algebra that the inverse of a matrix is equal to the adjoint matrix divided by the determinant of the matrix, the process of matrix inversion is completed. As shown in Figure 5 , each three-order symmetric matrix of the diagonal block is set as A, and it is assumed that the calculation in the figure is divided into four stages.

[0111] S25, if it is judged that the cost function value satisfies the iteration condition, the change amount of the Rodrigues rotation vector is received and step S21 is returned, if it is judged that the cost function value does not satisfy the iteration condition, the trust region related parameters are updated;

[0112] Specifically, if the change amount of the Rodrigues rotation vector can satisfy the iteration condition, step S21 is returned, otherwise the trust region related parameters are updated.

[0113] As shown in Figure 6 , the flow chart of the rotation optimization iteration structure is shown, the input parameter t is the maximum number of iterations; the parameter p is a long vector composed of the rotation vectors of all cameras, the dimension of which is 3n×1, where n is the number of cameras; J is the corresponding Jacobian matrix, that is, the partial derivative of the projection function with respect to the camera rotation vector, which is obtained by using the analytical differentiation method; g represents the gradient.

[0114] S26, steps S21-S25 are looped until it is judged that the iteration condition is exceeded, and the rotation matrix calculated by the current iteration number is the global rotation matrix.

[0115] Specifically, the condition for the algorithm to terminate in the iteration process is:

[0116] 1. The infinite norm of the gradient is less than ε1;

[0117] 2. The change Δp of p is less than ε2||p||;

[0118] 3. The iteration number k is greater than or equal to the maximum iteration number t.

[0119] In the whole algorithm optimization iteration process, the specific setting of the above parameters ε1, ε2 and t can avoid the result of the optimization rotation falling into a local optimal solution near the initial value, and in general cases, the parameters are set as ε1=ε2=10 -12 , t=15. At the same time, the size of the parameters can be changed according to whether the optimization result falls into a local optimal solution.

[0120] In the case where the algorithm satisfies the iteration judgment condition, the next equation solving is performed and it is judged whether the cost function is reduced, so as to decide whether to accept the updated Δp, and at the same time, the parameters such as ε1 and μ can be selected and the trust region is adjusted for the next iteration calculation.

[0121] S3. Complete the 3D reconstruction based on the global rotation matrix.

[0122] Specifically, the global matrix is ​​applied to the subsequent global translation calculation. The accuracy of the global translation calculation is affected by the rotation matrix. Then, triangulation is performed based on the feature points to generate three-dimensional points. Finally, all camera poses and coordinate points in three-dimensional space are nonlinearly optimized using the bundle adjustment method to achieve three-dimensional reconstruction.

[0123] As a preferred embodiment of this method, the matrix inversion method specifically includes:

[0124] Calculate each J based on the parametric equations. T The adjoint matrix A of the block matrix corresponding to J+μI * The first row of the matrix, a, b, and c, is stored in a FIFO for subsequent calculation of the determinant, while the calculated adjoint matrix is ​​stored in another FIFO.

[0125] Calculate the determinant of each block matrix A based on the elements in the adjoint matrix and find its reciprocal to obtain the reciprocal of the determinant of matrix A, |A|. -1 ;

[0126] Specifically, the values ​​of a, b, and c are read from the FIFO and multiplied and added to obtain the determinant of matrix A.

[0127] Multiplying the adjoint matrix by the inverse of the determinant of matrix A yields the corresponding inverse matrix A. -1 .

[0128] Specifically, considering the symmetric nature of the matrix, only the six elements of the upper triangle need to be calculated, which saves 33% of resource utilization.

[0129] Depend on Figure 7 It can be seen that the rotation optimization method of the present invention can optimize the global rotation estimate after the rotation averaging process. The optimization process can converge with fewer iterations, providing better accuracy for the subsequent three-dimensional reconstruction process.

[0130] In summary, the FPGA hardware accelerator for rotation optimization in 3D reconstruction proposed in this invention has the following advantages:

[0131] 1. The estimation of rotation does not require knowledge of translation and 3D structure, which can achieve decoupling because when the camera observes five or more identical points, their relative rotation can be independent of translation and 3D point structure.

[0132] 2. Unlike the rotational averaging process, the FPGA hardware accelerator uses image measurements as direct input during the optimization process, which can improve accuracy and extend the application from two views to multiple views.

[0133] 3. Parallel computing is used in the FPGA hardware circuit to complete the entire algorithm iterative solution process. In order to increase the concurrency of the program, without significantly increasing hardware resources, pipeline optimization can make the same operation in multiple iterations be executed sequentially, which has the effect of lower power consumption and faster operation compared with simple software implementation.

[0134] 4. FPGA hardware accelerators can be used after the rotation averaging process in 3D reconstruction, which can improve the accuracy of rotation estimation; at the same time, they can also solve the problem of pure rotation optimization and improve the robustness of the system.

[0135] like Figure 8 As shown, a 3D reconstruction system based on an FPGA hardware accelerator includes:

[0136] The data preprocessing module is used to acquire image observation data and perform preprocessing to obtain preprocessed data;

[0137] The rotation optimization module, based on an FPGA hardware accelerator, performs rotation optimization on the preprocessed data and estimates the global rotation matrix.

[0138] The reconstruction module is used to perform 3D reconstruction based on the global rotation matrix.

[0139] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0140] A 3D reconstruction device based on FPGA hardware accelerator:

[0141] At least one processor;

[0142] At least one memory for storing at least one program;

[0143] When the at least one program is executed by the at least one processor, the at least one processor implements the 3D reconstruction method based on an FPGA hardware accelerator as described above.

[0144] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0145] A storage medium, wherein processor-executable instructions are stored, and the processor-executable instructions are configured to implement the FPGA hardware accelerator-based three-dimensional reconstruction method according to any one of the above method embodiments when executed by a processor.

[0146] The above method embodiments are applicable to the present storage medium embodiment, the present storage medium embodiment specifically implements the same functions as the above method embodiments, and achieves the same beneficial effects as the above method embodiments.

[0147] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application. These equivalent modifications or replacements are also included in the scope defined by the claims of the present application.

Claims

1. A three-dimensional reconstruction method based on FPGA hardware accelerator, characterized in that, Includes the following steps: S1. Acquire image observation data and perform preprocessing to obtain preprocessed data; S2. Based on the FPGA hardware accelerator, the preprocessed data is rotated and optimized to estimate the global rotation matrix; S3. Complete the 3D reconstruction based on the global rotation matrix; The step of acquiring image observation data and preprocessing it to obtain preprocessed data specifically includes: S11. Acquire image observation data and classify and transform the image observation data to obtain data of each category; S12. Allocate data memory for each category of data and store it in the on-chip dual-port RAM of the FPGA hardware accelerator; S13. Complete the calculations in the cost function that are independent of the rotation parameters, including the sum of the projection products of the normalized coordinates of the points jointly observed by each pair of cameras onto any two coordinate axes; S14. Obtain preprocessed data; The step of optimizing the preprocessed data by rotation and estimating the global rotation matrix based on the FPGA hardware accelerator specifically includes: S21. Convert the Rodriguez vector in the preprocessed data into a rotation matrix; S22. Based on the preprocessed data, the minimum eigenvalue is calculated using the rotation matrix to obtain the cost function value; S23. Calculate the Jacobian matrix based on the analytical differential calculation method; S24. Construct parametric equations based on the Jacobian matrix and solve them using the matrix inversion method to obtain the change in the Rodriguez rotation vector. S25. If the cost function value satisfies the iteration condition, receive the change in the Rodriguez rotation vector and return to step S21. If the cost function value does not satisfy the iteration condition, update the relevant parameters of the trust domain. S26. Repeat steps S21-S25 until the iteration condition is exceeded. The rotation matrix calculated using the current iteration number is the global rotation matrix. The formula for calculating the rotation matrix is ​​as follows: In the above formula, Let r = [r1, r2, r3] denote the rotation matrix. T Represents the Rodriguez vector. This represents the transpose of the rotation vector. Let r1, r2, and r3 represent the rotation angles, and r1, r2, and r3 represent the projections of the rotation vector onto the mutually orthogonal x-axis, y-axis, and z-axis, respectively. The step of calculating the minimum eigenvalue based on preprocessed data and the rotation matrix to obtain the cost function value specifically includes: Based on the preprocessed data, the symmetric matrix is ​​calculated using the rotation matrix. Calculate the remaining related functions based on the elements in the symmetric matrix; Solve for the eigenvalues ​​based on the remaining relevant functions; The cost function value is obtained by summing the eigenvalues. The formula for calculating the symmetric matrix is ​​as follows: In the above formula, This represents the number of points jointly observed between any two cameras. This represents the i-th point. This represents the normalized camera coordinates of the point within one of the cameras. Represents the rotation matrix. This represents the normalized camera coordinates of the point on another camera; The formula for calculating the Jacobian matrix is ​​as follows: In the above formula, f x and f y X' represents the camera focal length obtained from the linear perspective law, and X', Y', and Z' represent the three-dimensional coordinates of the three-dimensional point in the camera coordinate system. The parametric equation is expressed as follows: In the above formula, Represents the Jacobian matrix. Indicates the radius of the trust domain. Represents the identity matrix. This represents the change in the rotation vector. Represents the gradient; The matrix inversion method specifically includes: Calculate the adjoint matrix based on the parametric equations; Calculate the determinant of the block matrix based on the elements in the adjoint matrix and find its reciprocal to obtain the reciprocal of the determinant of the block matrix. Multiplying the adjoint matrix and the inverse of the determinant of the block matrix yields the corresponding inverse matrix.

2. A 3D reconstruction system based on an FPGA hardware accelerator, characterized in that, A method for performing a 3D reconstruction based on an FPGA hardware accelerator as described in claim 1 includes: The data preprocessing module is used to acquire image observation data and perform preprocessing to obtain preprocessed data; The rotation optimization module, based on an FPGA hardware accelerator, performs rotation optimization on the preprocessed data and estimates the global rotation matrix. The reconstruction module is used to perform 3D reconstruction based on the global rotation matrix.

Images