An underwater binocular vision system refraction parameter self-calibration method

Abstract

This invention belongs to the field of underwater vision measurement technology and relates to a self-calibration method for refractive parameters of an underwater binocular vision system. The method first constructs a spatial equation based on Snell's law, including the water refractive index, the refractive interface normal vector, the interface distance, and the ray path coefficient. It then constructs a generalized state vector including environmental parameters, structural parameters, and path parameters. Utilizing the non-plane ray intersection constraint of binocular vision, it establishes the system residual equation and performs a nonlinear iterative least-squares solution by constructing a generalized Jacobian matrix to achieve adaptive calibration of the underwater vision system. This invention can simultaneously calculate the water refractive index and system structural parameters without requiring prior knowledge of water parameters or high-precision mechanical calibration, achieving adaptive calibration of the underwater vision system and significantly improving the accuracy and robustness of three-dimensional measurements in complex aquatic environments.

Description

Technical Field

[0001] This invention belongs to the field of underwater visual measurement technology, specifically relating to a method for self-calibrating the refractive parameters of an underwater binocular vision system. Background Technology

[0002] Current underwater binocular vision measurements typically employ sealed, waterproof housings to protect the camera. Light undergoes two refractions when passing through multi-layered planar interfaces such as water-glass-air or water-acrylic-air, causing traditional binocular vision models established in air to fail. To address this issue, research has proposed refractive imaging models based on Snell's law; however, these models generally assume that the refractive index of water is a known constant and treat the interface normal and interface position as fixed parameters obtained through calibration.

[0003] However, in actual marine or freshwater environments, the refractive index of water changes dynamically due to factors such as temperature, salinity, pressure, and wavelength. Using a fixed value would introduce significant systematic errors. Furthermore, installation deviations of the waterproof housing, thermal expansion and contraction, or mechanical deformation may cause deviations between the actual refractive interface normal vector and the preset value, or slight changes in the distance from the interface to the camera's optical center. These unmodeled parameter deviations will also reduce measurement accuracy.

[0004] Currently, there is a lack of a self-calibration method that can simultaneously estimate the refractive index of water, the interface normal vector, and the interface distance without requiring prior water parameters or precise installation and calibration. Therefore, there is an urgent need to propose a robust, system-level joint estimation technique for refractive parameters that can adapt to complex aquatic environments. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for self-calibrating the refractive parameters of an underwater binocular vision system.

[0006] To achieve the objectives of this invention, the following technical solutions are adopted.

[0007] A method for self-calibrating refractive parameters of an underwater binocular vision system, the underwater binocular vision system comprising a first camera and a second camera encapsulated in a waterproof housing; refracted light rays originating from the measurement target sequentially pass through a second refractive interface and a first refractive interface to form images on the imaging planes of the first and second cameras, comprising the following steps:

[0008] S1. Define the coordinate systems of the first camera and the second camera, and choose either one as the reference coordinate system. Calculate the coordinates of the first and second intersection points of any measurement target, expressed by the first estimated parameter, as well as the unit direction vectors of the first and second refracted rays, using Snell's law and the principle of ray tracing. Construct the spatial equations of the first and second refracted rays and set the coefficients of the spatial equations as the second estimated parameter. Wherein: the first estimated parameter includes the refractive index of the water body, the unit normal vector of the plane refractive interface, and the distance from the optical center of the first or second camera to the first refractive interface.

[0009] S2. Concatenate the first parameter to be estimated and the second parameter to be estimated to form the generalized state vector to be estimated.

[0010] S3. Based on the spatial equations of the first and second refracted rays, establish the residual vector of any measurement target containing the generalized state vector to be estimated, and stack the residual vectors of all targets to form the system residual equation.

[0011] S4. Based on each residual term in the system residual equation, construct the generalized Jacobian matrix by solving the partial derivatives of each parameter in the generalized state vector to be estimated. Iteratively update the generalized state vector to be estimated using a nonlinear least squares algorithm until the magnitude of the correction vector is less than the preset convergence threshold. Simultaneously solve the water refractive index, the unit normal vector of the plane refractive interface, and the distance from the optical center of the first camera or the second camera to the first refractive interface.

[0012] Furthermore, the first camera and the second camera are arranged in parallel.

[0013] Furthermore, the number m of the measurement targets is greater than or equal to 4.

[0014] Furthermore, the dimension of the second parameter to be estimated is 2m.

[0015] Furthermore, the total dimension of the generalized state vector to be estimated is 2m+4.

[0016] Furthermore, the first refractive interface is the interface between glass or acrylic and air; the second refractive interface is the interface between water and glass or acrylic.

[0017] Furthermore, the first intersection point is the first refracted ray from the first camera. The intersection point with the second refractive interface, whose coordinates are represented by the first parameter to be estimated, is marked as... The second intersection point is the second refracted ray from the second camera. The intersection point with the second refractive interface, whose coordinates are represented by the first parameter to be estimated, is marked as... .

[0018] Furthermore, the spatial equations for the first and second refracted rays are:

[0019] ;

[0020] In the formula: It is refracted light. The spatial equation; It is refracted light. The spatial equation; and The equation coefficients are the second parameter to be estimated, and each target corresponds to two equation coefficients to be estimated. and Let each represent a refracted ray represented by the first parameter to be estimated. and Unit direction vector.

[0021] Furthermore, the generalized state vector to be estimated is represented as:

[0022]

[0023] In the formula: r represents the generalized state vector to be estimated, which is 1×(2m+4) dimensional; Indicates the refractive index of water; Let represent the unit normal vector of the refractive interface, where: This represents the projection length of the unit normal vector of the refractive interface onto the x-axis of the reference coordinate system. d represents the projection length of the unit normal vector of the refraction interface onto the y-axis of the reference coordinate system; d represents the distance from the optical center of the first or second camera to the first refraction interface. The value represents the coefficient of the equation to be estimated corresponding to the target. The superscript indicates the target number, the subscript 1 indicates that the coefficient corresponds to the first refracted ray equation, and the subscript 2 indicates that the coefficient corresponds to the second refracted ray equation; T indicates the transpose of the matrix.

[0024] Furthermore, the residual vector Represented as:

[0025] ;

[0026] System residual equation Represented as:

[0027] ;

[0028] In the formula: the superscript indicates the target sequence number.

[0029] Furthermore, the generalized Jacobian matrix Let be a function of the generalized state vector, expressed as:

[0030] ;

[0031] In the formula: Represent the system residual equation, This represents the generalized state vector to be estimated, with the superscript indicating the target index. Represents the residual vector. Indicates the refractive index of water. This represents the projection length of the unit normal vector of the refractive interface onto the x-axis of the reference coordinate system. The projection length of the unit normal vector of the refractive interface onto the y-axis of the reference coordinate system is given by d, where d represents the distance from the optical center of the first or second camera to the first refractive interface. and These are the equation coefficients of the spatial equations for the first and second refracted rays.

[0032] Furthermore, the specific process of the iterative update includes the following steps:

[0033] S91. Calculate the correction vector of the generalized state vector.

[0034] ;

[0035] in, Let be the generalized Jacobian matrix in the current state. The system residual vector in the current state, with superscript... Indicates matrix transpose, superscript Represents the generalized inverse matrix;

[0036] S92. Update the generalized state vector. ;

[0037] S93. Judgment Whether it is valid, among which Set a preset convergence threshold; if the threshold is not met, repeat steps S91 and S92; if the threshold is met, output the current generalized state vector. ;in: 10 -6 ~10 -8 .

[0038] Beneficial effects: This invention aims to solve the systematic error problem caused by the variation of water refractive index and the uncertainty of the geometric parameters of the refractive interface in existing underwater binocular vision measurement. It provides a self-calibration method for system refractive parameters that does not require external calibration and does not rely on prior water information, so as to achieve high-precision and adaptive underwater 3D reconstruction. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of an underwater binocular vision measurement system. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0041] As an embodiment of the present invention, such as Figure 1 As shown, this embodiment employs an underwater binocular vision measurement system, including a first camera, a second camera, a planar parallel waterproof viewing window (made of glass or acrylic), and an image acquisition unit. The two cameras are rigidly fixed within the same waterproof housing, with their optical axes parallel. Light rays originate from the underwater target point P, passing sequentially through water, a glass (or acrylic) layer, and air. After two refractions, they are imaged onto the imaging planes of the left and right cameras, respectively. The refraction interfaces are the first refraction interface (air-solid interface) and the second refraction interface (water-solid refraction interface).

[0042] The known conditions in the system include: the establishment of the coordinate system of the two cameras, the intrinsic parameters of the two cameras, the extrinsic parameters from the second camera to the first camera, the air refractive index, the solid refractive index, the solid layer thickness, and the pixel coordinates of the target point on the imaging planes of the first and second cameras.

[0043] The system refractive parameters that need to be estimated include: the refractive index of the water body. , unit normal vector of the refraction interface The distance d from the optical center of the first camera to the first refractive interface. The refractive index of each target point.

[0044] The following steps provide a complete and feasible process.

[0045] Reconstruct the optical path model. Using the first camera coordinate system as the reference coordinate system, construct the optical path model based on Snell's law and the principle of ray tracing. For any target P within the field of view, according to the principles of geometric optics: on the first camera side, the ray originates from the optical center, passes through the first refractive interface, and the intersection of the refracted ray with the second refractive interface is denoted as... On the second camera side, the light ray originates from the optical center, passes through the first refractive interface, and the intersection of the refracted ray and the second refractive interface is denoted as... .

[0046] According to Snell's law, the refracted light rays after entering the water can be calculated. unit direction vector and refracted light unit direction vector At this point, the equations of the two refracted rays in the underwater space can be expressed as:

[0047] ;

[0048] in, It is refracted light. The spatial equation, It is refracted light. The spatial equation, and These are the coefficients of the refracted ray equations corresponding to the first and second cameras. These two coefficients are unknown and need to be estimated for each target point. and Let these represent the coordinates of the first and second intersection points, expressed using the first parameter to be estimated. and Let each represent a refracted ray represented by the first parameter to be estimated. and Unit direction vector. , , and The specific form can be obtained through symbolic computation based on Snell's law and the principle of ray tracing, using the steps disclosed in the literature (Guanqing Li et al., Underwater Refractive Stereo Vision Measurement and Simulation Imaging Model Based on Optical Path, Journal of Marine Science and Engineering, 2024, pp. 3-14).

[0049] Constructing the generalized state vector to be estimated. To achieve system-level self-calibration, this invention defines a generalized state vector r that includes environmental parameters, structural parameters, and path parameters. Assume there are a total of [missing information - likely parameters] in the field of view. If there are 3 matching non-collinear target points, then the generalized state vector r contains 1 / 2 * ... One element to be estimated:

[0050]

[0051] In the formula: r represents the generalized state vector to be estimated, which is 1×(2m+4) dimensional; Indicates the refractive index of water; Let represent the unit normal vector of the refractive interface, where: This represents the projection length of the unit normal vector of the refractive interface onto the x-axis of the reference coordinate system. d represents the projection length of the unit normal vector of the refraction interface onto the y-axis of the reference coordinate system; d represents the distance from the optical center of the first camera to the first refraction interface. The value represents the coefficient of the equation to be estimated corresponding to the target. The superscript indicates the target number, the subscript 1 indicates that the coefficient corresponds to the first refracted ray equation, and the subscript 2 indicates that the coefficient corresponds to the second refracted ray equation; T indicates the transpose of the matrix.

[0052] Establish the residual vector and system residual equations. Ideally, the refracted rays from the first and second cameras should intersect precisely at the target point P, i.e. Considering measurement noise and parameter bias, a residual vector is constructed.

[0053] For a single target point, the residual vector Defined as the distance vector between two skew rays in space:

[0054] ;

[0055] For all For each target point, their respective residual vectors are stacked to form the overall system residual equation. :

[0056] ;

[0057] The superscript indicates the target sequence number. It is A dimensional vector.

[0058] Constructing the generalized Jacobian matrix. To solve the nonlinear least squares problem for the residual equations of the system, it is necessary to construct the generalized Jacobian matrix. For the system residual equation Each term in the generalized state vector represents a different state. Find the partial derivatives of each parameter in the matrix. Generalized Jacobian matrix. The structure is as follows:

[0059] ;

[0060] In the formula: Represent the system residual equation, This represents the generalized state vector to be estimated, with the superscript indicating the target index. Represents the residual vector. Indicates the refractive index of water. This represents the projection length of the unit normal vector of the refractive interface onto the x-axis of the reference coordinate system. Let d represent the projection length of the unit normal vector of the refractive interface onto the y-axis of the reference coordinate system, and let d represent the distance from the optical center of the first camera to the first refractive interface. and These are the equation coefficients of the spatial equations for the first and second refracted rays.

[0061] To ensure that the generalized state vector has a unique solution, the number of equations must be greater than or equal to the number of unknowns, i.e. Solving for Therefore, in this embodiment, at least four non-collinear feature points need to be selected as observation targets.

[0062] The generalized state vector is solved iteratively. The iterative least squares method is used, and the specific process is as follows.

[0063] Initial value settings:

[0064] The initial value of the water refractive index is set to ;

[0065] The initial value of the unit normal vector of the refraction interface is set to ;

[0066] The initial value of distance d is taken as the mechanical design value of the waterproof shell;

[0067] Based on the aforementioned initial environment and structural parameters, the underwater binocular vision measurement algorithm disclosed in the literature (Guanqing Li et al., Underwater Refractive Stereo Vision Measurement and Simulation Imaging Model Based on Optical Path, Journal of Marine Science and Engineering, 2024, pp. 3-14) was used to numerically calculate the light coefficient corresponding to each target. and take it as and The initial value of .

[0068] All the above initial values ​​constitute the initial generalized state vector. .

[0069] Iterative updates:

[0070] Calculate the constant vector (i.e., the current residual) in the current state. ;

[0071] Calculate the generalized Jacobian matrix in the current state. ;

[0072] Calculate the correction vector of the generalized state vector , where superscript To represent the transpose of a matrix, the superscript... Represents the generalized inverse matrix;

[0073] Update the generalized state vector .

[0074] Convergence criteria:

[0075] Set convergence threshold The range is ;

[0076] judge Is it true? If not, use the updated version. Repeat the above steps; if true, stop the iteration, and the current... This is the optimal estimate.

[0077] Target 3D coordinates calculation. After iterative convergence, the final estimated coordinates are used. , d and each target and Substitute into the spatial equation of the refracted ray and Due to measurement errors, the two light rays may not intersect precisely. Therefore, the midpoint of the common perpendicular of the two skew lines is taken as the final three-dimensional coordinates of the target. :

[0078] ;

[0079] Repeat this step to obtain all High-precision three-dimensional coordinates of each target point.

[0080] Through the above steps, this embodiment not only completed the three-dimensional reconstruction of the target, but also simultaneously completed the self-calibration of the water refractive index, interface normal vector and interface distance, effectively eliminating the systematic errors caused by changes in the water environment and hardware deformation.

[0081] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for self-calibrating refractive parameters of an underwater binocular vision system, wherein the underwater binocular vision system includes a first camera and a second camera encapsulated in a waterproof housing; refracted light rays originate from the measurement target, sequentially pass through a second refractive interface and a first refractive interface, and are imaged onto the imaging planes of the first camera and the second camera, characterized in that: The steps include the following: S1. Define the coordinate systems of the first camera and the second camera, and select either one as the reference coordinate system. Calculate the coordinates of the first and second intersection points of any measurement target, expressed by the first estimated parameter, as well as the unit direction vectors of the first and second refracted rays, using Snell's law and the principle of ray tracing. Construct the spatial equations of the first and second refracted rays, and set the coefficients of the spatial equations as the second estimated parameter. Wherein: the first estimated parameter includes the refractive index of the water body, the unit normal vector of the plane refractive interface, and the distance from the optical center of the first or second camera to the first refractive interface. S2. Concatenate the first parameter to be estimated and the second parameter to be estimated to form the generalized state vector to be estimated. S3. Based on the spatial equations of the first and second refracted rays, establish the residual vector of any measurement target containing the generalized state vector to be estimated, and stack the residual vectors of all targets to form the system residual equation. S4. Based on each residual term in the system residual equation, construct the generalized Jacobian matrix by solving the partial derivatives of each parameter in the generalized state vector to be estimated. Iteratively update the generalized state vector to be estimated using a nonlinear least squares algorithm until the magnitude of the correction vector is less than the preset convergence threshold. Simultaneously solve the water refractive index, the unit normal vector of the plane refractive interface, and the distance from the optical center of the first camera or the second camera to the first refractive interface.

2. The method for self-calibrating refractive parameters of an underwater binocular vision system according to claim 1, characterized in that: The first camera and the second camera are set in parallel.

3. The method for self-calibrating refractive parameters of an underwater binocular vision system according to claim 2, characterized in that: The number of measurement targets m is greater than or equal to 4; the dimension of the second parameter to be estimated is 2m; the total dimension of the generalized state vector to be estimated is 2m+4.

4. The method for self-calibrating refractive parameters of an underwater binocular vision system according to claim 3, characterized in that: The first refractive interface is the interface between glass or acrylic and air; the second refractive interface is the interface between water and glass or acrylic.

5. The method for self-calibrating refractive parameters of an underwater binocular vision system according to claim 4, characterized in that: The first intersection point is the first refracted ray from the first camera. The intersection point with the second refractive interface, whose coordinates are represented by the first parameter to be estimated, is marked as... The second intersection point is the second refracted ray from the second camera. The intersection point with the second refractive interface, whose coordinates are represented by the first parameter to be estimated, is marked as... .

6. The method for self-calibrating refractive parameters of an underwater binocular vision system according to claim 5, characterized in that: The spatial equations for the first and second refracted rays are: ； In the formula: It is refracted light. The spatial equation; It is refracted light. The spatial equation; and The equation coefficients are the second parameter to be estimated, and each target corresponds to two equation coefficients to be estimated. and Let each represent a refracted ray represented by the first parameter to be estimated. and Unit direction vector.

7. The method for self-calibrating refractive parameters of an underwater binocular vision system according to claim 6, characterized in that: The generalized state vector to be estimated is represented as follows: In the formula: r represents the generalized state vector to be estimated, which is 1×(2m+4) dimensional; Indicates the refractive index of water; Let represent the unit normal vector of the refractive interface, where: This represents the projection length of the unit normal vector of the refractive interface onto the x-axis of the reference coordinate system. d represents the projection length of the unit normal vector of the refraction interface onto the y-axis of the reference coordinate system; d represents the distance from the optical center of the first or second camera to the first refraction interface. The value represents the coefficient of the equation to be estimated corresponding to the target. The superscript indicates the target number, the subscript 1 indicates that the coefficient corresponds to the first refracted ray equation, and the subscript 2 indicates that the coefficient corresponds to the second refracted ray equation; T indicates the transpose of the matrix.

8. The method for self-calibrating refractive parameters of an underwater binocular vision system according to claim 7, characterized in that: The residual vector Represented as: ； System residual equation Represented as: ； In the formula: the superscript indicates the target sequence number.

9. The method for self-calibrating refractive parameters of an underwater binocular vision system according to claim 8, characterized in that: The generalized Jacobian matrix Let be a function of the generalized state vector to be estimated, expressed as: ； In the formula: Represent the system residual equation, This represents the generalized state vector to be estimated, with the superscript indicating the target index. Represents the residual vector. Indicates the refractive index of water. This represents the projection length of the unit normal vector of the refractive interface onto the x-axis of the reference coordinate system. The projection length of the unit normal vector of the refractive interface onto the y-axis of the reference coordinate system is given by d, where d represents the distance from the optical center of the first or second camera to the first refractive interface. and These are the equation coefficients of the spatial equations for the first and second refracted rays.

10. The method for self-calibrating refractive parameters of an underwater binocular vision system according to claim 9, characterized in that: The specific process of the iterative update includes the following steps: S101. Calculate the correction vector of the generalized state vector to be estimated. ； in, Let be the generalized Jacobian matrix in the current state. The system residual vector in the current state, with superscript Indicates matrix transpose, superscript Represents the generalized inverse matrix; S102. Update the generalized state vector to be estimated. ; S103, Judgment Whether it is valid, among which Set a preset convergence threshold; if the threshold is not met, repeat steps S101 and S102; if the threshold is met, output the current generalized state vector to be estimated. ;in: 10 -6 ~10 -8 .

Images