A camera layout method for 3D reconstruction of ship waves based on numerical simulation

Abstract

The present invention provides a camera layout method in three-dimensional reconstruction of ship waves based on numerical simulation, which relates to the field of ship and ocean engineering, and includes the following steps: by parameterizing the virtual camera in the physical engine, a mathematical model of a single virtual camera posture is obtained, and the mathematical model of the single virtual camera posture is uniformly defined for multiple cameras to obtain a parameterized camera layout mathematical model; for the mathematical model of the camera layout and the parallax angle of the camera layout, the camera layout scheme is obtained in combination with the influence of the camera layout on the sparse reconstruction and dense reconstruction of ship waves when the viewing distance is changed. The present invention parameterizes the posture of a single camera and establishes a unified mathematical model to describe the layout of multiple cameras. Based on the camera layout, a better camera layout scheme is obtained, and the accuracy of the three-dimensional reconstruction of ship waves and the number of spatial points under the camera layout scheme are significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of ship and ocean engineering, and in particular to a camera layout method in three-dimensional reconstruction of ship traveling waves based on numerical simulation. Background Art

[0002] In the process of 3D reconstruction, the camera layout reflects the position and posture of a single camera when shooting the target object, as well as the relative posture between multiple cameras. Camera layout is a key factor affecting 3D reconstruction. Under the same conditions, a reasonable camera layout can obtain a more accurate reconstruction result.

[0003] The existing technology performs sparse reconstruction of the images obtained by numerical simulation of ship waves, which is independent of the camera shooting order, and calculates the depth map and fuses it into an overall dense reconstruction, obtaining a more realistic reconstruction result and restoring the waveform characteristics of ship waves.

[0004] However, currently no parameterized mathematical model is used to describe and optimize the camera layout. As a result, it is easy for the same number of cameras to cause difficulty in stably controlling the quality of 3D reconstruction due to unstable shooting methods, or for the unreasonable camera posture to result in too few feature matches to perform accurate triangulation. Summary of the invention

[0005] In view of this, the purpose of the present invention is to propose a camera layout method in the three-dimensional reconstruction of ship waves based on numerical simulation. By building a stereoscopic vision simulation environment in a physical engine, the three-dimensional wave surface of the ship waves obtained by the viscous flow CFD numerical simulation is subjected to multi-perspective sparse reconstruction of the ship waves and dense reconstruction of the ship waves based on depth map fusion. On this basis, the influence of different camera layouts on the three-dimensional reconstruction of ship waves is studied. The lack of research on camera layout schemes in the three-dimensional reconstruction of ship waves under the existing technology is solved, the camera layout parameterization is realized, and different camera layouts are compared horizontally, which significantly improves the rationality of the camera layout and the accuracy of the three-dimensional reconstruction of ship waves.

[0006] The technical means adopted by the present invention are as follows:

[0007] A camera layout method for three-dimensional reconstruction of ship waves based on numerical simulation comprises the following steps:

[0008] By parametrically defining the virtual camera in the physical engine, a mathematical model of a single virtual camera posture is obtained, and the mathematical model of the single virtual camera posture is uniformly defined for multiple cameras to obtain a parameterized camera layout mathematical model;

[0009] Based on the mathematical model of the camera layout, combined with the influence of the camera layout on the sparse reconstruction and dense reconstruction of ship waves when the parallax angle is changed, the optimal parallax angle is obtained;

[0010] According to the mathematical model of the camera layout and the parallax angle of the camera layout, combined with the influence of the camera layout on the sparse reconstruction and dense reconstruction of the ship wave when the viewing distance is changed, a camera layout solution is obtained.

[0011] Furthermore, the steps of obtaining the camera layout mathematical model are:

[0012] Parametric definition: for a virtual camera in the physical engine, a dynamic Euler angle mathematical model is selected to define a local coordinate system. The virtual camera is located on a spherical surface with the origin of the local coordinate system as the center of the sphere. The main optical axis of the virtual camera passes through the origin. The position of the virtual camera is represented by the dynamic Euler rotation angle of the coordinate axis to obtain a mathematical model of a single virtual camera posture.

[0013] All virtual cameras are parameterized to obtain mathematical models of the poses of all virtual cameras. In the mathematical models of the poses of all virtual cameras, multiple virtual cameras are evenly distributed on the same sphere. The parallax angle and viewing distance are used to represent the camera layout to obtain a parameterized mathematical model of the camera layout.

[0014] Furthermore, the steps of obtaining the camera layout parallax angle are as follows:

[0015] Changing the parallax angle of the camera layout mathematical model to obtain the effect of the camera layout on the sparse reconstruction of ship waves when the parallax angle is changed;

[0016] The influence of the parallax angle on the sparse reconstruction of ship waves is determined by a parallax angle determination method, and a variation range of the parallax angle of sparse reconstruction is obtained;

[0017] The parallax angle variation range is further densely reconstructed, and the integrity of the scene and the distribution of noise during dense reconstruction are compared and analyzed to obtain the optimal parallax angle.

[0018] Furthermore, the step of obtaining the camera layout viewing distance is:

[0019] According to the camera layout mathematical model and the optimal parallax angle, the viewing distance of the camera layout mathematical model is continuously changed to obtain the effect of the camera layout on the sparse reconstruction of ship waves when the viewing distance is changed;

[0020] By using a sight distance determination method, the influence of the variable sight distance on the sparse reconstruction of ship waves is determined to obtain a sight distance variation range of the sparse reconstruction;

[0021] The viewing distance variation range is further densely reconstructed, and the scene integrity and the distribution of noise points during dense reconstruction are compared and analyzed to obtain the optimal viewing distance. Combined with the optimal parallax angle, the final camera layout solution is output.

[0022] Furthermore, the parallax angle determination method is: selecting a parallax angle with more spatial points and less reprojection error as the parallax angle variation range.

[0023] Furthermore, the viewing distance determination method is: selecting a viewing distance with more spatial points and less reprojection error as the viewing distance variation range.

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] The present invention provides a camera layout method in ship wave three-dimensional reconstruction based on numerical simulation. The influence of different camera layouts on ship wave three-dimensional reconstruction is studied. The problem of lack of research on camera layout schemes in ship wave three-dimensional reconstruction in the prior art is solved, and the camera layout parameterization is realized. Multiple cameras are evenly arranged on a spherical surface, and the main optical axes of the cameras pass through the center of the sphere. The radius of the spherical surface is the viewing distance and the angle between the main optical axes is the parallax angle. The camera layout is adjusted by coordinating the two main parameters of the viewing distance and the parallax angle, and a high-quality camera layout scheme is found therefrom. Different camera layouts are compared horizontally, which significantly improves the rationality of the camera layout and the accuracy of ship wave three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0027] Figure 1 The figure is a flow chart of the method of the present invention.

[0028] Figure 2 This is the mathematical model diagram of the camera pose.

[0029] Figure 3 is a parameterized camera layout diagram.

[0030] Figure 4 A diagram showing the effect of changing the parallax angle on the number of sparsely reconstructed spatial points.

[0031] Figure 5 Graph showing the effect of changing the parallax angle on the reprojection error of sparse reconstruction.

[0032] Figure 6 Graph showing the effect of changing the parallax angle on dense reconstruction.

[0033] Figure 7 This is the dense reconstruction result image with a parallax angle of 6 degrees.

[0034] Figure 8This is the dense reconstruction result image with a parallax angle of 7 degrees.

[0035] Fig. 9 This is the dense reconstruction result image with a parallax angle of 8 degrees.

[0036] Fig.10 A diagram showing the effect of changing the viewing distance on the number of sparsely reconstructed spatial points.

[0037] Fig.11 Graph showing the impact of changing viewing distance on the reprojection error of sparse reconstruction.

[0038] Fig.12 Graph showing the effect of changing the viewing distance on dense reconstruction.

[0039] Fig.13 This is the dense reconstruction result image with a viewing distance of 9m.

[0040] Fig.14 This is the dense reconstruction result image with a viewing distance of 10m.

[0041] Fig.15 This is the dense reconstruction result image with a viewing distance of 11m. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0043] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0044] The present invention provides a technical solution: a camera layout method in three-dimensional reconstruction of ship waves based on numerical simulation, which is as follows:

[0045] (1) Parameterized camera layout mathematical model

[0046] Reasonable arrangement of the camera orientation can significantly improve the accuracy of sparse reconstruction. Sparse reconstruction is the basis of subsequent work. In order to better complete the three-dimensional reconstruction of ship waves, the present invention studies the camera layout.

[0047] In an ideal virtual environment, the camera layout is studied and the ship's waves are observed by multiple virtual cameras. In order to make the poses of each virtual camera have a unified parameterized expression, the present invention uses a dynamic Euler angle mathematical model to describe the camera poses, such as Figure 2 shown.

[0048] Figure 2 Where R is the distance between the optical center of the camera and the origin of the local coordinate system; the principal optical axis of the camera passes through the origin of the local coordinate system; ∠1 is the angle (°) of rotation of the camera around the y-axis; ∠2 is the angle (°) of rotation of the camera around the x1-axis; the x1-axis is obtained by rotating the x-axis around the y-axis by ∠1.

[0049] The pose parameters of each camera in the camera array are shown in Table 1.

[0050] Table 1 Camera layout parameters

[0051] Distance(m) ∠1(°) ∠2(°) Distance(m) ∠1(°) ∠2(°) View 1 R 2θ 2θ View 14 R -1θ 0 View 2 R 1θ 2θ View 15 R -2θ 0 View 3 R 0 2θ View 16 R 2θ -1θ View 4 R -1θ 2θ View 17 R 1θ -1θ View 5 R -2θ 2θ View 18 R 0 -1θ View 6 R 2θ 1θ Perspective 19 R -1θ -1θ View 7 R 1θ 1θ View 20 R -2θ -1θ View 8 R 0 1θ View 21 R 2θ -2θ View 9 R -1θ 1θ View 22 R 1θ -2θ View 10 R -2θ 1θ View 23 R 0 -2θ View 11 R 2θ 0 View 24 R -1θ -2θ View 12 R 1θ 0 View 25 R -2θ -2θ View 13 R 0 0

[0052] In the case of the layout in Table 1, the camera array in the simulation environment is as follows Figure 3 shown.

[0053] In this layout, the virtual cameras are arranged evenly on the same sphere. By changing the angle θ and radius R, the overall camera layout can be changed. When the spherical radius R is the same, the larger the angle θ, the larger the parallax angle between the cameras. Therefore, the change in angle θ can reflect the impact of the parallax angle between the cameras on the sparse reconstruction results. When the angle θ is the same, the larger the radius R, the larger the line of sight between the camera and the ship's wave. Therefore, R can reflect the impact of the line of sight on the sparse reconstruction results.

[0054] (2) Dense reconstruction of numerical simulation of ship waves

[0055] On the basis of sparse reconstruction, the global neighborhood perspective is calculated for each perspective through the global perspective selection algorithm; the sparse spatial points obtained in the sparse reconstruction process are used as seed points to construct patches, and the depth value optimization algorithm of the patches is executed to obtain the depth map of the seed points; the depth map of the seed points is expanded through the regional growing method, and the pixel points within the neighborhood range are added to the optimization queue to obtain a complete depth map; all the depth maps are fused into a whole to obtain the final dense spatial points.

[0056] Dense reconstruction can obtain richer spatial point information, realistically restore three-dimensional spatial points based on two-dimensional images, and intuitively calculate the waveform fluctuation characteristics of ship waves. However, dense reconstruction is based on sparse reconstruction, and the camera layout has a very obvious impact on the quality of sparse reconstruction and dense reconstruction.

[0057] (3) Camera layout parallax angle

[0058] The influence of parallax angle change on the 3D reconstruction of ship waves is studied by changing θ. The present invention first studies the influence of the change of θ in the range of 0° to 25° with an interval of 1° on the sparse reconstruction. The results are as follows: Figure 4 , Figure 5 shown.

[0059] In the range of 5° to 15°, there are more spatial points and the reprojection error is relatively small, while outside the range, the number of spatial points is small when θ is small or large.

[0060] When the parallax angle increases, the difference in image acquisition between the binocular camera models increases, resulting in a decrease in the number of paired points for feature matching and thus a decrease in the number of spatial points; at the same time, the increase in the parallax angle increases the baseline length between the cameras, thereby reducing the projection error during triangulation, making the coordinate calculation of the spatial points more accurate, thereby obtaining a smaller reprojection error.

[0061] When the parallax angle is large, the number of paired points obtained by feature matching is small, and even cannot meet the requirements of solving the triangulation equation, and the number of spatial points obtained is low.

[0062] When the parallax angle decreases, the consistency of the shooting results between cameras increases, the number of paired points increases, and more spatial points are obtained; at the same time, the camera baseline is shorter, so the triangulation error is larger, which makes the reprojection error larger.

[0063] When the parallax angle is close to 0, the consistency between the cameras is too strong and the camera baseline is too short to meet the calculation input requirements of triangulation, and the spatial point cannot be calculated.

[0064] After analysis, the present invention selects data within the range of 5° to 15° for further dense reconstruction, and the obtained results are as follows: Figure 6 shown.

[0065] When the parallax angle is 7°, the number of spatial points obtained by dense reconstruction reaches a peak. Figure 4 , Figure 5 It can be found that the sparse reconstruction result with a parallax angle of 7° also has a large number of spatial points and a small reprojection error, so it is a more ideal value in this result. The reconstruction results of five cases with parallax angles of 5°, 6°, 7°, 8°, and 9° are selected for further analysis.

[0066] The results of sparse reconstruction have obtained relatively uniform spatial points, and several waveform contours of ship waves can be seen. The results of dense reconstruction have obtained a clear and relatively complete surface in the ship wave area, and the main contours of the ship waves can be seen. The waveform characteristics are obvious, and a more realistic model is intuitively reconstructed. However, the results of dense reconstruction also have similar defects, that is, there are noise points in the non-ship wave area, and there are holes in the wake of the ship waves. Relatively speaking, when θ=6°, there are fewer noise points in the non-ship wave area, but a small amount of noise points are generated inside the hole in the wake wave area. When θ=7°, there are more noise points in the non-ship wave area, and the inside of the hole is clear and free of noise points. Figure 7 , Figure 8 , Fig. 9 shown.

[0067] Based on the above analysis, θ=7° is a relatively ideal reconstruction result, in which the number of sparse spatial points is large, the reprojection error of sparse spatial points is small, and the number of dense spatial points is also large. Therefore, the present invention fixes θ=7° and continues the camera layout test.

[0068] (4) Camera layout viewing distance

[0069] The influence of the change of line of sight on the three-dimensional reconstruction of ship waves is studied by changing R. The present invention studies the influence of the change of R on the sparse reconstruction when the interval is 1m in the range of 1m to 25m. The results are as follows Fig.10 , Fig.11 shown.

[0070] The quality of the sparse reconstruction results is better when the viewing distance is in the range of 5m to 15m. At this time, the number of spatial points is large and the reprojection error is also small.

[0071] When the viewing distance increases, the virtual camera's ability to collect detailed information about objects decreases, the observed object only occupies a smaller area in the image, the number of feature points obtained by feature detection decreases, resulting in a decrease in the number of calculated spatial points, and an increase in the reprojection error, which reduces the robustness of the overall calculation results.

[0072] When the viewing distance is small, although the camera can capture more detailed images, it can only capture part of the object, the ability to capture the entire object is reduced, and the reconstruction is insufficiently complete, resulting in a smaller number of calculated spatial points.

[0073] Therefore, the present invention selects data within the range of 5m to 15m to further perform dense reconstruction, and the obtained results are as follows Fig.12 shown.

[0074] When the viewing distance is 9m, the number of spatial points obtained by dense reconstruction reaches a peak. Fig.10 , Fig.11It can be found that the number of spatial points in the sparse reconstruction result when the viewing distance is 9m is relatively large and the reprojection error is relatively small, so it is a relatively ideal value in this result. The reconstruction results of five cases with viewing distances of 7m, 8m, 9m, 10m, and 11m are selected for further analysis.

[0075] When R = 9m, although the number of sparse space points is large and the reprojection error is small, there are obvious outliers. Relatively speaking, the sparse space points when R = 10m are more uniform. When R = 7m and R = 11m, the integrity of the wake wave area is relatively good, and the voids are relatively unobvious, but there are relatively more noise points in the non-ship wave area. Although the ship wave waveform characteristics obtained by dense reconstruction are obvious, the reconstruction results also have consistent and similar defects. Fig.13 , Fig.14 , Fig.15 shown.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A camera layout method for three-dimensional reconstruction of ship waves based on numerical simulation, characterized in that: The steps include: By parametrically defining the virtual camera in the physical engine, a mathematical model of a single virtual camera posture is obtained, and the mathematical model of the single virtual camera posture is uniformly defined for multiple cameras to obtain a parameterized camera layout mathematical model; The steps for obtaining the camera layout mathematical model are: Parametric definition: for a virtual camera in the physical engine, a dynamic Euler angle mathematical model is selected to define a local coordinate system. The virtual camera is located on a spherical surface with the origin of the local coordinate system as the center of the sphere. The main optical axis of the virtual camera passes through the origin. The position of the virtual camera is represented by the dynamic Euler rotation angle of the coordinate axis to obtain a mathematical model of a single virtual camera posture. All virtual cameras are parameterized to obtain mathematical models of the positions and postures of all virtual cameras. In the mathematical models of the positions and postures of all virtual cameras, multiple virtual cameras are evenly distributed on the same sphere. The parallax angle and the viewing distance are used to represent the camera layout, and a parameterized mathematical model of the camera layout is obtained. The steps for obtaining the sight distance are: According to the camera layout mathematical model and the optimal parallax angle, the viewing distance of the camera layout mathematical model is continuously changed to obtain the effect of the camera layout on the sparse reconstruction of ship waves when the viewing distance is changed; By using a sight distance determination method, the influence of the variable sight distance on the sparse reconstruction of ship waves is determined to obtain a sight distance variation range of the sparse reconstruction; Further densely reconstruct the viewing distance variation range, compare and analyze the scene integrity and the distribution of noise during dense reconstruction, obtain the optimal viewing distance, and output the final camera layout solution in combination with the optimal parallax angle; Based on the mathematical model of the camera layout, combined with the influence of the camera layout on the sparse reconstruction and dense reconstruction of ship waves when the parallax angle is changed, the optimal parallax angle is obtained; The steps for obtaining the optimal parallax angle are: Changing the parallax angle of the camera layout mathematical model to obtain the effect of the camera layout on the sparse reconstruction of ship waves when the parallax angle is changed; The influence of the parallax angle on the sparse reconstruction of ship waves is determined by a parallax angle determination method, and a variation range of the parallax angle of sparse reconstruction is obtained; Further densely reconstructing the parallax angle variation range, comparing and analyzing the integrity of the scene and the distribution of noise points during dense reconstruction, and obtaining the optimal parallax angle; According to the mathematical model of the camera layout and the parallax angle of the camera layout, combined with the influence of the camera layout on the sparse reconstruction and dense reconstruction of the ship wave when the viewing distance is changed, a camera layout solution is obtained.

2. The camera layout method in the three-dimensional reconstruction of ship waves based on numerical simulation according to claim 1, characterized in that: The parallax angle determination method is: selecting a parallax angle with more spatial points and less reprojection error as the parallax angle variation range.

3. The camera layout method in the three-dimensional reconstruction of ship waves based on numerical simulation according to claim 1, characterized in that: The viewing distance determination method is: selecting a viewing distance with more spatial points and less reprojection error as the viewing distance variation range.

Images