Underwater binocular system three-dimensional reconstruction method and system, and terminal equipment

Abstract

The invention discloses a three-dimensional reconstruction method for an underwater binocular system, and relates to the technical field of underwater optical three-dimensional measurement. The method comprises the following steps: receiving camera internal parameters and distortion coefficients calibrated by left and right cameras in the underwater binocular system in the air and relative external parameters between the two cameras; receiving plane equation parameters of a cabin mirror outer plane of the underwater binocular system in a left camera coordinate system and synchronous left and right image pairs of an object to be measured acquired by the underwater binocular system in an underwater environment, wherein the plane equation parameters comprise a normal vector n and a distance d from an original point to the plane; preprocessing the acquired left and right image pairs, and calculating a disparity map or a feature point corresponding relation by adopting a stereo matching algorithm; and a virtual image pre-calibration method is adopted to calibrate the initial error of the point cloud before and after correction, and the focusing parameter of the camera is adjusted. According to the method, cabin mirror interface parameters are accurately calibrated, strict light path tracking and the Snell law are combined, a physically accurate multi-refraction correction model is constructed, high-precision correction is carried out on the basis of traditional binocular reconstruction, and finally accurate three-dimensional coordinates of a real underwater object are obtained.

Description

Technical Field

[0001] This invention relates to the field of underwater optical three-dimensional measurement technology, specifically to a three-dimensional reconstruction method, system, and terminal equipment for an underwater binocular system. Background Technology

[0002] Underwater binocular stereo vision is an important non-contact measurement method for acquiring three-dimensional shape information of underwater objects. It is widely used in marine resource exploration, underwater engineering structure inspection, underwater archaeology, aquatic biological research, and environmental perception and operation guidance of underwater robots such as ROVs / AUVs. Its basic principle is to simulate the parallax of human binoculars through two sets of imaging optical paths and use the principle of triangulation to recover three-dimensional spatial information from two-dimensional images.

[0003] However, underwater binocular systems typically require placing the camera inside a waterproof pressure chamber and observing through a flat glass or acrylic window (the chamber mirror). When light travels from the water medium through the chamber mirror to the camera, it undergoes two refractions at the "water-glass-air" interface, severely violating the assumptions of traditional pinhole camera models and the principles of binocular triangulation. Directly applying parameters calibrated in air for underwater 3D reconstruction introduces systematic errors that vary with the object's position, leading to distorted point clouds, scale distortion, and decreased accuracy.

[0004] Existing technologies attempt to address this problem, mainly falling into two categories: one is refraction model correction based on precise physical parameters, and the other is approximation through equivalent models calibrated underwater. The former requires precise knowledge of the binoculars' geometric parameters (thickness, refractive index) and spatial attitude, which is difficult to guarantee in engineering assembly, and the model is complex and computationally intensive. The latter (such as virtual camera models) simplifies calculations, but the calibration process is cumbersome, accuracy is limited by the calibration range and calibration plate precision, and the model's generalization ability is limited. Therefore, this invention proposes a three-dimensional reconstruction method, system, and terminal device for underwater binocular systems. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and terminal device for three-dimensional reconstruction of an underwater binocular system. By accurately calibrating the interface parameters of the binoculars and combining strict optical path tracing and Snell's law, a physically accurate multi-refractive correction model is constructed, thereby performing high-precision correction on the basis of traditional binocular reconstruction, and finally obtaining the accurate three-dimensional coordinates of real underwater objects.

[0006] According to a first aspect of the present invention, in order to achieve the above-mentioned objective, the present invention provides the following technical solution: a method for three-dimensional reconstruction of an underwater binocular system, comprising the following steps: S1. Receive the camera intrinsic parameters, distortion coefficients and relative extrinsic parameters between the left and right cameras calibrated in the air in the underwater binocular system, where the relative extrinsic parameters include the rotation matrix R and the translation vector T; S2. Receive the plane equation parameters of the outer plane of the underwater binocular system in the left camera coordinate system and the synchronous left and right image pairs of the object to be measured acquired by the underwater binocular system in the underwater environment. The plane equation parameters include the normal vector n and the distance d from the origin to the plane. S3. Preprocess the acquired left and right image pairs, use a stereo matching algorithm to calculate the disparity map or feature point correspondence, and reconstruct the initial 3D point cloud P of the measured object in the left camera coordinate system based on the traditional binocular triangulation principle. initial ; S4. For the initial 3D point cloud P initial Any point P in i Construct lines connecting the optical centers of the left and right cameras to form a spatial straight line L in the left camera coordinate system. L And the spatial straight line L in the right camera coordinate system R ; S5. Based on the image point coordinates and the camera parameters calibrated in step S1, perform reverse optical path tracing for the left and right cameras respectively: calculate the incident direction of the light ray originating from the camera's optical center on the inner surface of the capsule. According to Snell's law and the plane equation parameters calibrated in step S2, calculate the first refraction direction of the light ray at the air-glass interface on the inner surface of the capsule and the second refraction direction at the glass-water interface on the outer surface of the capsule, and finally obtain the underwater light ray direction v. water The image point coordinates p are constructed. L underwater light direction v water The deterministic calculation relationship, namely the point cloud refraction correction model; S6. Using the relative extrinsic parameters calibrated in step S1, transform the underwater ray equation of the right camera to the coordinate system of the left camera, thereby obtaining two underwater ray equations corresponding to the same object point in the left camera coordinate system, specifically the left camera ray equation and the right camera ray equation, wherein the left camera ray equation is based on O L and v water_L The right camera's ray equation is based on O R and v water_R ; S7. Calculate the optimal intersection point P between the two light rays from the left and right cameras using the least squares method. corrected (X c ,Y c Z c Using the 3D point coordinates corrected by the multi-refractive model, traverse all matching point pairs and repeat steps S4 to S8 to obtain the final corrected 3D point cloud {P}. corrected}; S8. Based on the established point cloud refraction correction model, conduct a pre-calibration test of the virtual image of the underwater object under test, determine the initial error of the point cloud before and after correction, adjust the camera focusing parameters, and recalibrate the camera intrinsic parameters, distortion coefficients, relative extrinsic parameters between the two cameras, and the outer plane equation of the underwater binocular system's telescope.

[0007] Furthermore, the underwater binocular vision system includes two cameras, left and right, sealed within a waterproof housing, and a front-facing planar mirror.

[0008] Furthermore, the calibration of the plane equation parameters is achieved using a counter-assisted binocular measurement system, specifically including: A set of auxiliary binocular camera systems with known intrinsic and relative extrinsic parameters is set up directly opposite the underwater binocular system. Use a three-dimensional calibration object with coded points affixed to its surface, and use photogrammetry to measure the precise three-dimensional coordinates of its coded points in the global coordinate system; The calibration object is placed in the common field of view of the two systems, and the three-dimensional coordinates of the coding point in their respective coordinate systems are reconstructed by the underwater binocular system and the auxiliary binocular system, respectively. Based on the same-named coding points, calculate the coordinate transformation relationship from the auxiliary stereo system coordinate system to the underwater stereo system coordinate system; The feature points attached to the outer surface of the underwater binocular system are reconstructed in three dimensions using an auxiliary binocular system. The coordinates of the feature points are then transformed into the coordinate system of the underwater binocular system for plane fitting, thereby obtaining the plane equation parameters.

[0009] Furthermore, in step S3, a stereo matching algorithm is used to calculate the disparity map or the correspondence between feature points. Based on the traditional binocular triangulation principle, the initial 3D point cloud P of the measured object in the left camera coordinate system is reconstructed. initial The details are as follows: For a pair of matching points, its three-dimensional coordinates are obtained by solving the following equation: (1) (2) Where S L S R The scale factor is used. This calculation process does not consider the refraction of light at the cabin mirror, so the result is the initial coordinates containing errors.

[0010] Furthermore, the construction of the point cloud refraction correction model is specifically as follows: Let the equation of the mirror plane be n·P = d, where P is any point on the plane, and the direction from the optical center O is v. air The incident ray intersects the plane at point P. glass_in According to Snell's law, the direction of light rays refracted into the glass is v. glasssatisfy: (3) Where n air The refractive index of air is denoted by ×, which represents the vector product. Similarly, at the glass-water interface (assuming the inner and outer surfaces are parallel and the normal vector is n), from direction v glass The direction of the refracted light rays entering the water, v water satisfy: (4) v is calculated based on the image point coordinates (u, v) and the intrinsic parameter K. air Then, the final underwater ray direction v is calculated recursively. water .

[0011] Furthermore, in step S7, the optimal intersection point of the two spatial rays is calculated using the least squares method, specifically as follows: Let the equation of the light ray from the left camera be P = O L + t L v L The equation of the right camera ray, now transformed to the left coordinate system, is P = O. R + t R ՛ v R ՛, where v L and v R ՛ is the normalized direction vector, t L and t R It is a scalar parameter; Finding parameter t L and t R Such that two points P L = O L + t L v L With P R = O R ՛+ t R v R Minimizing the distance between ՛ transforms the problem into solving a system of linear equations. The optimal solution is the midpoint that minimizes the sum of squared distances, or a point obtained by solving the normal equations. The coordinates of this point are the corrected 3D point P. corrected .

[0012] Furthermore, the initial 3D point cloud P of the measured object in the left camera coordinate system is reconstructed. initial After that, it is also necessary to process the initial point cloud P. initial Perform outlier removal or filtering operations.

[0013] According to a second aspect of the present invention, the present invention provides an underwater binocular system three-dimensional reconstruction system for implementing the underwater binocular system three-dimensional reconstruction method described in the first aspect, comprising: The underwater binocular vision acquisition component includes an underwater binocular camera system for completing the calibration of the binoculars parameters, a waterproof housing, and an illumination device; Underwater transport and positioning components, specifically ROVs, AUVs, or fixed underwater platforms, are used to carry data acquisition modules and control their movement or orientation relative to the object being measured. Data processing and reconstruction components, specifically including: The parameter calibration module is used to receive the camera intrinsic parameters, distortion coefficients and relative extrinsic parameters between the left and right cameras calibrated in the air in the underwater binocular system. The relative extrinsic parameters include the rotation matrix R and the translation vector T. The image acquisition module is used to receive the plane equation parameters of the outer plane of the underwater binocular system in the coordinate system of the left camera and the synchronous left and right image pairs of the object to be measured acquired by the underwater binocular system in the underwater environment. The plane equation parameters include the normal vector n and the distance d from the origin to the plane. The initial 3D point cloud construction module is used to preprocess the acquired left and right image pairs, calculate the disparity map or feature point correspondence using a stereo matching algorithm, and reconstruct the initial 3D point cloud P of the measured object in the left camera coordinate system based on the traditional binocular triangulation principle. initial ; The spatial line construction module is used to construct an initial 3D point cloud P. initial Any point P in i Construct lines connecting the optical centers of the left and right cameras to form a spatial straight line L in the left camera coordinate system. L And the spatial straight line L in the right camera coordinate system R ; The point cloud refraction correction model construction module is used to perform reverse optical path tracing for the left and right cameras based on image point coordinates and calibrated camera parameters. It calculates the incident direction of light rays originating from the camera's optical center on the inner surface of the optics. Based on Snell's law and calibrated plane equation parameters, it sequentially calculates the first refraction direction at the air-glass interface on the inner surface of the optics and the second refraction direction at the glass-water interface on the outer surface of the optics, ultimately obtaining the underwater light ray direction v. water The image point coordinates p are constructed. L underwater light direction v water The deterministic calculation relationship, namely the point cloud refraction correction model; The underwater ray equation conversion module is used to convert the underwater ray equation of the right camera to the left camera coordinate system using calibrated relative extrinsic parameters. This results in two underwater ray equations corresponding to the same object point in the left camera coordinate system: the left camera ray equation and the right camera ray equation. The left camera ray equation is based on O... L and v water_L The right camera's ray equation is based on O R and v water_R ; The solution output module is used to calculate the optimal intersection point P between the two rays from the left and right cameras using the least squares method. corrected (X c , Y c Z c ), as the 3D point coordinates corrected by the multi-refractive model, traverse all matching point pairs to obtain the final corrected 3D point cloud {P corrected}; The error correction module is used to conduct a pre-calibration test of the virtual image of the underwater object based on the established point cloud refraction correction model, measure the initial error of the point cloud before and after correction, adjust the camera focusing parameters, and recalibrate the camera intrinsic parameters, distortion coefficients, relative extrinsic parameters between the two cameras, and the outer plane equation of the underwater binocular system's telescope.

[0014] According to a third aspect of the present invention, the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein when the processor loads and executes the computer program, it employs the underwater binocular system three-dimensional reconstruction method described in the first aspect.

[0015] According to a fourth aspect of the present invention, the present invention provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the underwater binocular system three-dimensional reconstruction method as described in the first aspect.

[0016] This invention has at least the following beneficial effects: 1. This invention provides a precise mathematical description of the two refractions in the underwater optical path using a rigorous ray tracing physical model (Snell's law), directly correcting the systematic errors introduced by refraction at the medium interface from the perspective of imaging principles. Compared to existing methods that use approximate equivalent models or empirical formulas, this method is based on real physical optical processes for correction, has a higher theoretical accuracy limit, and can significantly reduce the distortion and scale distortion of the reconstructed point cloud; 2. This invention creatively decomposes the complex problem of refraction error correction into two relatively independent stages: "scope interface parameter calibration" and "model calculation based on known parameters." For the difficult-to-measure-precise spatial attitude of the scope, a practical scheme for indirect, high-precision calibration using a counter-assisted measurement system is proposed. This method only requires calibration once during system setup or maintenance to obtain stable key physical parameters, eliminating the need for repeated, tedious calibration processes in subsequent large-scale underwater measurements, greatly enhancing the method's engineering practicality and operability.

[0017] 3. In the correction process, this invention utilizes two independent underwater rays from the same object point for spatial intersection calculation. By establishing joint constraints between the "end reflection ray path" and the "initial underwater ray path" of the left and right camera optical paths, and using the least squares method to solve for the optimal spatial intersection point, this method can effectively fuse dual-view information, smooth out the uncertainty of individual optical path calculation caused by image matching errors, camera calibration residuals, and environmental disturbances, thereby significantly improving the stability, reliability, and robustness of the final 3D point coordinates and the overall reconstruction results.

[0018] 4. After establishing the point cloud refraction correction model, this invention uses a pre-experiment to reconstruct the apparent virtual image of the underwater object, calibrates the error between the underwater virtual image and the real image of the object, and uses this error value to adjust the focusing parameters of the underwater binocular system. The virtual image pre-calibration method can eliminate the error in the initial value of the three-dimensional reconstruction correction caused by the virtual image point offset, further improving the accuracy and reliability of the final corrected three-dimensional point coordinates.

[0019] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0020] Figure 1 This is an overall flowchart of the three-dimensional reconstruction method described in this invention; Figure 2 This is a schematic diagram of the underwater binocular system refraction parameter calibration method in an embodiment of the present invention; Figure 3 This is a schematic diagram of the principle of the underwater binocular system multi-refractive correction model in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0022] Example 1: Please see Figures 1-3 This invention provides a technical solution: a method for three-dimensional reconstruction of an underwater binocular system, comprising the following steps: Step S1: Construct an underwater binocular vision system, including two cameras sealed inside a waterproof casing and a planar mirror at the front; calibrate in air to obtain the intrinsic parameter matrix K of each camera. L K R The distortion coefficients and the relative extrinsic parameters between the two cameras, namely the rotation matrix R and the translation vector T; Specifically, two industrial cameras are selected, their optical axes are fixed parallel or convergent, and they are installed in a custom-made waterproof housing with a flat glass lens mounted in front. In the laboratory, using a high-precision checkerboard calibration board and the Zhang Zhengyou calibration method, the intrinsic parameter matrix K of the left and right cameras is calibrated separately. L K R The lens distortion coefficients are then used for stereo calibration to obtain the rotation matrix R and translation vector T of the right camera coordinate system relative to the left camera coordinate system. This step is performed in the air to ensure the accuracy of the baseline parameters. Step S2: Using a three-dimensional calibration object and a counter-axis auxiliary measurement system, calibrate the plane equation parameters of the outer plane of the underwater binocular system's telescope in the left camera coordinate system, denoted as the normal vector n and the distance d from the origin to the plane, as follows: A set of auxiliary binocular camera systems with known intrinsic and relative extrinsic parameters is set up directly opposite the underwater binocular system. Use a three-dimensional calibration object with coded points affixed to its surface, and use photogrammetry to measure the precise three-dimensional coordinates of its coded points in the global coordinate system; The calibration object is placed in the common field of view of the two systems, and the three-dimensional coordinates of the coding point in their respective coordinate systems are reconstructed by the underwater binocular system and the auxiliary binocular system, respectively. Based on the same-named coding points, calculate the coordinate transformation relationship from the auxiliary stereo system coordinate system to the underwater stereo system coordinate system; The feature points pasted on the outer surface of the underwater binocular system are reconstructed in three dimensions using an auxiliary binocular system. The coordinates of the feature points are then transformed into the coordinate system of the underwater binocular system for plane fitting, thereby obtaining the plane equation parameters. Finally, the plane equation of the cabin mirror plane in the left camera coordinate system is fitted: n x X + n y Y + n z Z = d, and store the unit normal vector n and distance d, such as Figure 2 As shown; Step S3: Install the calibrated underwater binocular system on a carrier such as an ROV, AUV, or fixed bracket, and acquire synchronous left and right image pairs of the surface of the object under test in the underwater environment; The calibrated underwater binocular system is installed on the gimbal of the underwater robot (ROV). The ROV is operated to approach the target to be tested (such as a subsea pipeline or a ship hull weld). The attitude is adjusted to ensure that the target is within the binocular field of view, and the binocular camera is triggered to simultaneously acquire clear left and right image pairs. Step S4: Preprocess the acquired images (e.g., denoising, enhancement), use a stereo matching algorithm to calculate the disparity map or feature point correspondence, and reconstruct the initial 3D point cloud P of the measured object in the left camera coordinate system based on the traditional binocular triangulation principle. initial : Based on the traditional binocular triangulation principle, the initial 3D point cloud P of the measured object in the left camera coordinate system is reconstructed. initial The details are as follows: For a pair of matching points, its three-dimensional coordinates are obtained by solving the following equation: (1) (2) Where S L S R As a scale factor, this calculation process does not consider the refraction of light at the cabin mirror, so the result is the initial coordinates containing errors; Specifically, the intrinsic parameters, distortion coefficients, and extrinsic parameters (R, T) calibrated in step S1 are used to perform stereo correction on the image, aligning the left and right image rows. A semi-global matching (SGM) or feature-point-based matching algorithm is employed to calculate the disparity of each pixel or feature point. Based on the binocular triangulation formula, the three-dimensional coordinates of each matching point in the left camera coordinate system are calculated, forming the initial point cloud P. initial .like Figure 3 As shown, since refraction is not considered, this reconstructed point (the intersection of the dashed lines in the figure) is not the actual location of the object point; Step S5: For the initial point cloud P initial Any point P in i (X i Y i Z i In the left camera coordinate system, it is related to the optical center O of the left camera. L The lines connecting them form a spatial straight line L. L (End reflection optical path); Similarly, by using the right camera matching point and extrinsic parameters, its position relative to the right camera's optical center O in the right camera coordinate system can be obtained. R The spatial straight line L formed by the connecting lines R Two straight lines L L With L RIn theory, they should intersect at real object points; Specifically, such as Figure 3 As shown, for a point P in the initial point cloud i It is itself two lines of sight that do not consider refraction (from O) L To P i From O R To P i The incorrect intersection point P. This invention will use this point P as an example. i With left camera optical center O L The connecting line is considered as a spatial straight line L L (End-reflection optical path), its direction vector can be obtained through P i and O L The coordinates are calculated. Similarly, using the matching relationship and the extrinsic parameters of the right camera, the corresponding straight line L in the right camera coordinate system can be obtained. R ; Step S6: Perform reverse ray tracing for a single camera (taking the left camera as an example); for the left image point p L (u L , v L Its back-projected light rays travel in direction v inside the camera (in the air). air From the light core O L The emitted light undergoes a first refraction at the inner surface of the telescope (air-glass interface) and enters the glass; subsequently, it undergoes a second refraction at the outer surface of the telescope (glass-water interface) and enters the water, its direction changing to v. water The underwater ray and the straight line L in step S5 L Theoretically, they should point in the same direction; according to Snell's law and the known equation of the mirror plane (n, d), the glass refractive index n glass Water refractive index n water It is possible to establish the image point coordinates p L underwater light direction v water The deterministic calculation relationship, namely the point cloud refraction correction model; The construction of the point cloud refraction correction model is as follows: Let the equation of the mirror plane be n·P = d, where P is any point on the plane, and the direction from the optical center O is v. air The incident ray intersects the plane at point P. glass_in According to Snell's law, the direction of light rays refracted into the glass is v. glass satisfy: (3) Where n air The refractive index of air is denoted by ×, which represents the vector product. Similarly, at the glass-water interface (assuming the inner and outer surfaces are parallel and the normal vector is n), from direction v glassThe direction of the refracted light rays entering the water, v water satisfy: (4) v is calculated based on the image point coordinates (u, v) and the intrinsic parameter K. air Then, the final underwater ray direction v is calculated recursively. water ; Specifically, such as Figure 3 As shown, taking the left camera as an example, reverse optical path tracing is performed to generate P. i left image point p L (u L , v L According to the intrinsic parameter matrix K L It is possible to calculate its normalized direction vector v in the left camera coordinate system (the air portion). air This ray first reaches the inner surface of the capsule (air-glass interface). The equation of the capsule plane (n, d) and the optical center O are known. L The incident point P can be calculated. glass_in According to Snell's law, using the refractive index n of air... air and glass refractive index n glass The direction v of the refracted light rays inside the glass can be calculated. glass Since it is assumed that the inner and outer surfaces of the capsule are parallel (or their relationship is known), the ray continues to propagate to point P on the outer surface (glass-water interface). glass_out Applying Snell's law again, using the glass refractive index n... glass and the refractive index of water n water Calculate the direction v of the light ray that finally enters the water. water_L Thus, the system from image point p was established. L The actual direction of light underwater, v water_L The exact mapping relationship is then applied to the right camera, performing the exact same process to obtain the underwater ray direction v in the right camera coordinate system. water_R ; Step S7: Perform step S6 on both the left and right cameras to obtain two underwater ray equations described by their respective camera coordinate systems: the left camera ray equation is based on O... L and v water_L The right camera's ray equation is based on O R and v water_R Using the extrinsic parameters (R, T) calibrated in step S1, the ray equation of the right camera is uniformly transformed to the coordinate system of the left camera, resulting in O. R ՛ and v water_R ՛, specifically as follows: The optimal intersection point of two spatial rays is calculated using the least squares method, specifically as follows: Let the equation of the light ray from the left camera be P = O L + tL v L The equation of the right camera ray, now transformed to the left coordinate system, is P = O. R + t R ՛ v R ՛, where v L and v R ՛ is the normalized direction vector, t L and t R It is a scalar parameter; Finding parameter t L and t R Such that two points P L = O L + t L v L With P R = O R ՛+ t R v R Minimizing the distance between ՛ transforms the problem into solving a system of linear equations. The optimal solution is the midpoint that minimizes the sum of squared distances, or a point obtained by solving the normal equations. The coordinates of this point are the corrected 3D point P. corrected ; Step S8: In the left camera coordinate system, two spatial rays (from the left and right cameras respectively) may not intersect due to noise and model errors; the least squares method is used to calculate the optimal intersection point coordinates P between the two rays. corrected (X c Y c Z c This point is the high-precision 3D point coordinate after correction using the multi-refractive model; traverse all matching point pairs and repeat steps S5 to S8 to obtain the final corrected high-precision 3D point cloud {P}. corrected}, Specifically, such as Figure 3 As shown, there are now two spatial rays in the left camera coordinate system: Ray_L: P = O L +t L v water_L and Ray_R: P = O R ՛+ t R v water_R Due to various errors, these two light rays are usually skew and do not intersect. The least squares method is used to find their optimal intersection point, and the objective function is the sum of squares of the distances between corresponding points on the two light rays: min ||(O L + t L vwater_L ) - (O R ՛ + t R v water_R ՛) ||^2, by adjusting t L and t R Taking the derivative and setting it to zero transforms the problem into solving a system of linear equations to find the optimal value of t. L_opt and t R_opt . t L_op Substituting t into the Ray-L equation (or taking the midpoint of the two points), we can obtain the corrected high-precision three-dimensional point coordinates P. corrected Traverse all matching points and repeat steps S5-S8 to finally obtain a high-precision 3D point cloud {P} with refraction system errors eliminated. corrected}; Step S9: Using the established point cloud refraction correction model, conduct a virtual image pre-calibration test of the underwater object under test, measure the initial error of the point cloud before and after correction, adjust the camera focusing parameters, and recalibrate the camera intrinsic parameters, distortion coefficients, relative extrinsic parameters between the two cameras, and the outer plane equation of the underwater binocular system's telescope. Finally, use the virtual image pre-calibration method to obtain the new camera parameters.

[0023] In summary, the method and system provided in this application, by introducing precise calibration of the binoculars' parameters and a rigorous optical path tracing refraction model, transform the approximate process of traditional underwater binocular reconstruction into a precise calculation process based on the principles of physical optics, fundamentally improving the accuracy of underwater 3D reconstruction. This method combines theoretical rigor with engineering practicality, providing an effective technical means for high-precision underwater measurement applications.

[0024] Example 2: This embodiment provides an underwater binocular system 3D reconstruction system for implementing the underwater binocular system 3D reconstruction method described in Embodiment 1, including: The underwater binocular vision acquisition component includes an underwater binocular camera system for completing the calibration of the binoculars parameters, a waterproof housing, and an illumination device; Underwater transport and positioning components, specifically ROVs, AUVs, or fixed underwater platforms, are used to carry data acquisition modules and control their movement or orientation relative to the object being measured. Data processing and reconstruction components, specifically including: The parameter calibration module is used to receive the camera intrinsic parameters, distortion coefficients and relative extrinsic parameters between the left and right cameras calibrated in the air in the underwater binocular system. The relative extrinsic parameters include the rotation matrix R and the translation vector T. The image acquisition module is used to receive the plane equation parameters of the outer plane of the underwater binocular system in the coordinate system of the left camera and the synchronous left and right image pairs of the object to be measured acquired by the underwater binocular system in the underwater environment. The plane equation parameters include the normal vector n and the distance d from the origin to the plane. The initial 3D point cloud construction module is used to preprocess the acquired left and right image pairs, calculate the disparity map or feature point correspondence using a stereo matching algorithm, and reconstruct the initial 3D point cloud P of the measured object in the left camera coordinate system based on the traditional binocular triangulation principle. initial ; The spatial line construction module is used to construct an initial 3D point cloud P. initial Any point P in i Construct lines connecting the optical centers of the left and right cameras to form a spatial straight line L in the left camera coordinate system. L And the spatial straight line L in the right camera coordinate system R ; The point cloud refraction correction model construction module is used to perform reverse optical path tracing for the left and right cameras based on image point coordinates and calibrated camera parameters. It calculates the incident direction of light rays originating from the camera's optical center on the inner surface of the optics. Based on Snell's law and calibrated plane equation parameters, it sequentially calculates the first refraction direction at the air-glass interface on the inner surface of the optics and the second refraction direction at the glass-water interface on the outer surface of the optics, ultimately obtaining the underwater light ray direction v. water The image point coordinates p are constructed. L underwater light direction v water The deterministic calculation relationship, namely the point cloud refraction correction model; The underwater ray equation conversion module is used to convert the underwater ray equation of the right camera to the left camera coordinate system using calibrated relative extrinsic parameters. This results in two underwater ray equations corresponding to the same object point in the left camera coordinate system: the left camera ray equation and the right camera ray equation. The left camera ray equation is based on O... L and v water_L The right camera's ray equation is based on O R and v water_R ; The solution output module is used to calculate the optimal intersection point P between the two rays from the left and right cameras using the least squares method. corrected (X c , Y c Z c ), as the 3D point coordinates corrected by the multi-refractive model, traverse all matching point pairs to obtain the final corrected 3D point cloud {P corrected}; The error correction module is used to conduct a pre-calibration test of the virtual image of the underwater object based on the established point cloud refraction correction model, measure the initial error of the point cloud before and after correction, adjust the camera focusing parameters, and recalibrate the camera intrinsic parameters, distortion coefficients, relative extrinsic parameters between the two cameras, and the outer plane equation of the underwater binocular system's telescope.

[0025] Example 3: The present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor. When the processor loads and executes the computer program, it employs the underwater binocular system three-dimensional reconstruction method described in Embodiment 1.

[0026] It should be noted that the terminal device can be a computer device such as a desktop computer, a laptop computer, or a cloud server, and the terminal device includes, but is not limited to, a processor and a memory. For example, the terminal device may also include input / output devices, network access devices, and buses.

[0027] Furthermore, the processor can be a central processing unit (CPU). Of course, depending on the actual use, other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be used. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it in this regard.

[0028] Example 4: The present invention provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the underwater binocular system three-dimensional reconstruction method described in Embodiment 1.

[0029] Computer programs can be stored on computer-readable media. Computer programs include computer program code, which can be in the form of source code, object code, executable files, or certain middleware. Computer-readable media include any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that computer-readable media includes, but is not limited to, the aforementioned components. It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0030] For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances. When an element is referred to as being "assembled on," "mounted on," "fixed to," or "set on" another element, it may be directly on the other element or there may be an intermediate element present. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element present. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible embodiments.

[0031] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0032] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

Claims

1. A method for three-dimensional reconstruction of an underwater binocular system, characterized in that, Includes the following steps: S1. Receive the camera intrinsic parameters, distortion coefficients and relative extrinsic parameters between the left and right cameras calibrated in the air in the underwater binocular system, where the relative extrinsic parameters include the rotation matrix R and the translation vector T; S2. Receive the plane equation parameters of the outer plane of the underwater binocular system in the left camera coordinate system and the synchronous left and right image pairs of the object to be measured acquired by the underwater binocular system in the underwater environment. The plane equation parameters include the normal vector n and the distance d from the origin to the plane. S3. Preprocess the acquired left and right image pairs, use a stereo matching algorithm to calculate the disparity map or feature point correspondence, and reconstruct the initial 3D point cloud P of the measured object in the left camera coordinate system based on the traditional binocular triangulation principle. initial ; S4. For the initial 3D point cloud P initial Any point P in i Construct lines connecting the optical centers of the left and right cameras to form a spatial straight line L in the left camera coordinate system. L And the spatial straight line L in the right camera coordinate system R ; S5. Based on the image point coordinates and the camera parameters calibrated in step S1, perform reverse optical path tracing for the left and right cameras respectively: calculate the incident direction of the light ray originating from the camera's optical center on the inner surface of the capsule. According to Snell's law and the plane equation parameters calibrated in step S2, calculate the first refraction direction of the light ray at the air-glass interface on the inner surface of the capsule and the second refraction direction at the glass-water interface on the outer surface of the capsule, and finally obtain the underwater light ray direction v. water The image point coordinates p are constructed. L underwater light direction v water The deterministic calculation relationship, namely the point cloud refraction correction model; S6. Using the relative extrinsic parameters calibrated in step S1, transform the underwater ray equation of the right camera to the coordinate system of the left camera, thereby obtaining two underwater ray equations corresponding to the same object point in the left camera coordinate system, specifically the left camera ray equation and the right camera ray equation, wherein the left camera ray equation is based on O L and v water_L The right camera's ray equation is based on O R and v water_R ; S7. Calculate the optimal intersection point P between the two light rays from the left and right cameras using the least squares method. corrected (X c Y c Z c Using the 3D point coordinates corrected by the multi-refractive model, traverse all matching point pairs and repeat steps S4 to S8 to obtain the final corrected 3D point cloud {P}. corrected }; S8. Based on the established point cloud refraction correction model, conduct a pre-calibration test of the virtual image of the underwater object under test, determine the initial error of the point cloud before and after correction, adjust the camera focusing parameters, and recalibrate the camera intrinsic parameters, distortion coefficients, relative extrinsic parameters between the two cameras, and the outer plane equation of the underwater binocular system's telescope.

2. The underwater binocular system three-dimensional reconstruction method according to claim 1, characterized in that: The underwater binocular vision system includes two cameras, left and right, sealed in a waterproof shell, and a front-facing planar mirror.

3. The underwater binocular system three-dimensional reconstruction method according to claim 2, characterized in that: The calibration of the plane equation parameters is achieved using a counter-assisted binocular measurement system, specifically including: A set of auxiliary binocular camera systems with known intrinsic and relative extrinsic parameters is set up directly opposite the underwater binocular system. Use a three-dimensional calibration object with coded points affixed to its surface, and use photogrammetry to measure the precise three-dimensional coordinates of its coded points in the global coordinate system; The calibration object is placed in the common field of view of the two systems, and the three-dimensional coordinates of the coding point in their respective coordinate systems are reconstructed by the underwater binocular system and the auxiliary binocular system, respectively. Based on the same-named coding points, calculate the coordinate transformation relationship from the auxiliary stereo system coordinate system to the underwater stereo system coordinate system; The feature points attached to the outer surface of the underwater binocular system are reconstructed in three dimensions using an auxiliary binocular system. The coordinates of the feature points are then transformed into the coordinate system of the underwater binocular system for plane fitting, thereby obtaining the plane equation parameters.

4. The underwater binocular system three-dimensional reconstruction method according to claim 3, characterized in that: In step S3, a stereo matching algorithm is used to calculate the disparity map or the correspondence of feature points. Based on the traditional binocular triangulation principle, the initial 3D point cloud P of the measured object in the left camera coordinate system is reconstructed. initial The details are as follows: For a pair of matching points, its three-dimensional coordinates are obtained by solving the following equation: （1） （2） Where S L S R The scale factor is used. This calculation process does not consider the refraction of light at the cabin mirror, so the result is the initial coordinates containing errors.

5. The underwater binocular system three-dimensional reconstruction method according to claim 4, characterized in that: The point cloud refraction correction model is constructed as follows: Let the equation of the mirror plane be n·P = d, where P is any point on the plane, and the direction from the optical center O is v. air The incident ray intersects the plane at point P. glass_in According to Snell's law, the direction of light rays refracted into the glass is v. glass satisfy: （3） Where n air The refractive index of air is denoted by ×, which represents the vector product. Similarly, at the glass-water interface (assuming the inner and outer surfaces are parallel and the normal vector is n), from direction v glass The direction of the refracted light rays entering the water, v water satisfy: （4） v is calculated based on the image point coordinates (u, v) and the intrinsic parameter K. air Then, the final underwater ray direction v is calculated recursively. water .

6. The underwater binocular system three-dimensional reconstruction method according to claim 5, characterized in that: In step S7, the optimal intersection point of the two spatial rays is calculated using the least squares method, specifically as follows: Let the equation of the light ray from the left camera be P = O L + t L v L The equation of the right camera ray, now transformed to the left coordinate system, is P = O. R + t R ՛ v R ՛, where v L and v R ՛ is the normalized direction vector, t L and t R It is a scalar parameter; Finding parameter t L and t R Such that two points P L = O L + t L v L With P R = O R ՛+ t R v R Minimizing the distance between ՛ transforms the problem into solving a system of linear equations. The optimal solution is the midpoint that minimizes the sum of squared distances, or a point obtained by solving the normal equations. The coordinates of this point are the corrected 3D point P. corrected .

7. The underwater binocular system three-dimensional reconstruction method according to claim 6, characterized in that: The initial 3D point cloud P of the measured object in the left camera coordinate system was reconstructed. initial After that, it is also necessary to process the initial point cloud P. initial Perform outlier removal or filtering operations.

8. A three-dimensional reconstruction system for an underwater binocular system, used to implement the three-dimensional reconstruction method for an underwater binocular system as described in any one of claims 1 to 7, characterized in that, include: The underwater binocular vision acquisition component includes an underwater binocular camera system for completing the calibration of the binoculars parameters, a waterproof housing, and an illumination device; Underwater transport and positioning components, specifically ROVs, AUVs, or fixed underwater platforms, are used to carry data acquisition modules and control their movement or orientation relative to the object being measured. Data processing and reconstruction components, specifically including: The parameter calibration module is used to receive the camera intrinsic parameters, distortion coefficients and relative extrinsic parameters between the left and right cameras calibrated in the air in the underwater binocular system. The relative extrinsic parameters include the rotation matrix R and the translation vector T. The image acquisition module is used to receive the plane equation parameters of the outer plane of the underwater binocular system in the coordinate system of the left camera and the synchronous left and right image pairs of the object to be measured acquired by the underwater binocular system in the underwater environment. The plane equation parameters include the normal vector n and the distance d from the origin to the plane. The initial 3D point cloud construction module is used to preprocess the acquired left and right image pairs, calculate the disparity map or feature point correspondence using a stereo matching algorithm, and reconstruct the initial 3D point cloud P of the measured object in the left camera coordinate system based on the traditional binocular triangulation principle. initial ; The spatial line construction module is used to construct an initial 3D point cloud P. initial Any point P in i Construct lines connecting the optical centers of the left and right cameras to form a spatial straight line L in the left camera coordinate system. L And the spatial straight line L in the right camera coordinate system R ; The point cloud refraction correction model construction module is used to perform reverse optical path tracing for the left and right cameras based on image point coordinates and calibrated camera parameters. It calculates the incident direction of light rays originating from the camera's optical center on the inner surface of the optics. Based on Snell's law and calibrated plane equation parameters, it sequentially calculates the first refraction direction at the air-glass interface on the inner surface of the optics and the second refraction direction at the glass-water interface on the outer surface of the optics, ultimately obtaining the underwater light ray direction v. water The image point coordinates p are constructed. L underwater light direction v water The deterministic calculation relationship, namely the point cloud refraction correction model; The underwater ray equation conversion module is used to convert the underwater ray equation of the right camera to the left camera coordinate system using calibrated relative extrinsic parameters. This results in two underwater ray equations corresponding to the same object point in the left camera coordinate system: the left camera ray equation and the right camera ray equation. The left camera ray equation is based on O... L and v water_L The right camera's ray equation is based on O R and v water_R ; The solution output module is used to calculate the optimal intersection point P between the two rays from the left and right cameras using the least squares method. corrected (X c Y c Z c ), as the 3D point coordinates corrected by the multi-refractive model, traverse all matching point pairs to obtain the final corrected 3D point cloud {P corrected }; The error correction module is used to conduct a pre-calibration test of the virtual image of the underwater object based on the established point cloud refraction correction model, measure the initial error of the point cloud before and after correction, adjust the camera focusing parameters, and recalibrate the camera intrinsic parameters, distortion coefficients, relative extrinsic parameters between the two cameras, and the outer plane equation of the underwater binocular system's telescope.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor loads and executes the computer program, it employs the underwater binocular system three-dimensional reconstruction method as described in any one of claims 1 to 7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the underwater binocular system three-dimensional reconstruction method as described in any one of claims 1 to 7.

Images