Underwater camera imaging model and design and calibration method thereof

Abstract

The invention belongs to the technical field of computer vision and three-dimensional measurement, and particularly relates to an underwater camera imaging model and a design and calibration method thereof. The underwater camera imaging model comprises a camera, a sealing protective cover and sealing protective cover glass; the sealing protective cover glass is of a plane structure, and the influence of the glass thickness on an imaging light path cannot be ignored; the camera observes an underwater target through sealing protective cover glass in front of the sealing protective cover; when a target point D is given underwater, the imaging coordinates corresponding to the target point D can be obtained through the underwater camera imaging model according to the three-dimensional coordinates of the target point D in the middle coordinate system associated with the front surface of the sealing protective cover. According to the underwater camera imaging model provided by the invention, the general installation condition of the camera in the protective cover is considered, the optical axis of the camera is allowed to be not strictly vertical to the protective cover glass, the thickness of the protective cover glass is also considered, the application range is wider, and the imaging coordinates of spatial points can be accurately calculated.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and three-dimensional measurement, and in particular to an underwater camera imaging model and a design and calibration method thereof. Background Art

[0002] With the development of marine technology and economy, the demand for underwater observation and detection equipment is increasing. As an information acquisition sensor, the camera has many advantages such as direct, intuitive and efficient, and plays a very important role in underwater measurement tasks. One of the main challenges in the underwater measurement imaging process is the propagation characteristics of light in different media (air, water), especially the refraction problem of the water interface and the camera end protective cover interface. This causes the traditional measurement model and calibration method in the air to fail underwater, making underwater three-dimensional measurement difficult. One solution is to recalibrate the camera internal parameters underwater, and include the refraction effect of the protective cover and water on the light in the imaging distortion coefficient and the camera internal parameters. This method only approximates the real imaging model to a certain extent. When the target motion range increases, the deviation between the real imaging and the calculated imaging will also increase. Another method is to simplify the relative position relationship between the camera and the protective cover, such as assuming that the camera optical axis is perpendicular to the sealed protective cover glass, while ignoring the thickness of the protective cover glass, and then using the analytical solution to calculate the imaging coordinates corresponding to the spatial point. When the thickness of the protective cover glass is not negligible, or when the angle between the camera optical axis and the normal vector of the sealed protective cover glass is large, the adaptability of the model will decrease. Therefore, a modeling method that can overcome the influence of light refraction in underwater environments and achieve high-precision imaging is urgently needed. Summary of the invention

[0003] The purpose of the present invention is to provide a design method for an underwater camera imaging model, through which the imaging coordinates of a space target point D can be solved. Before the model is used in practical applications, it is necessary to calibrate and solve parameters such as the relative position relationship between the camera and the sealed protective glass, and the relative posture relationship between the intermediate coordinate system and the outer coordinate system of the sealed protective cover in the model. Therefore, based on the underwater camera imaging model, the sum of squares of the reprojection errors of the calibration reference points is set as the objective function, and the iterative optimization method is used to obtain the minimum value of the objective function, so as to complete the solution of the relative position relationship between the camera and the sealed protective glass, and the relative posture relationship between the intermediate coordinate system and the outer coordinate system of the sealed protective cover.

[0004] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: an underwater camera imaging model, comprising: a camera, a sealed protective cover, and a sealed protective cover glass;

[0005] The sealed protective cover glass is a planar structure, and the influence of the glass thickness on the imaging optical path cannot be ignored; the camera observes the underwater target through the sealed protective cover glass in front of the sealed protective cover; when a target point D underwater is given, the three-dimensional coordinates in the intermediate coordinate system associated with the front surface of the sealed protective cover can be obtained through the underwater camera imaging model.

[0006] The intermediate coordinate system is as follows: the centroid of the front surface of the sealed protective cover is used to project onto the outer surface of the sealed protective cover glass, the projection point is used as the origin of the intermediate coordinate system, the Z axis of the intermediate coordinate system is perpendicular to the outer surface of the sealed protective cover glass, and points to the outside of the sealed protective cover; when the operator who establishes the coordinate system maintains the same observation angle as the camera inside the sealed protective cover, the X axis of the intermediate coordinate system is horizontal to the right, and the Y axis is vertically downward.

[0007] A method for designing an underwater camera imaging model comprises the following steps:

[0008] 1) Determine the refractive index of the sealed protective cover glass, the refractive index of the medium outside the cover, and the thickness of the sealed protective cover glass;

[0009] 2) Establishing the inner coordinate system and the outer coordinate system of the sealing protective cover;

[0010] 3) Determine the relative position relationship T2 between the intermediate coordinate system and the outer coordinate system of the sealing protective cover;

[0011] 4) Obtain the relative position relationship T1 between the inner coordinate system of the sealing protective cover and the camera coordinate system;

[0012] 5) According to steps 1) to 4), the underwater camera imaging model is established, and the imaging coordinates corresponding to the given underwater target point D are obtained according to the underwater camera imaging model.

[0013] The establishment of the inner coordinate system of the protective cover and the outer coordinate system of the protective cover is specifically as follows:

[0014] 2-1) Establishing the inner coordinate system of the protective cover:

[0015] The origin of the coordinate system is the foot of a perpendicular line drawn from the optical center of the underwater camera to the inner surface of the sealed protective cover glass. The X-axis is the projection of the X-axis of the camera coordinate system onto the inner surface of the sealed protective cover glass. The Z-axis is the normal vector of the sealed protective cover surface, pointing to the outside of the sealed protective cover. The Y-axis direction satisfies the right-hand rule.

[0016] 2-2) Establishing the outer coordinate system of the sealing protective cover:

[0017] The origin of the coordinate system is the foot of a perpendicular line drawn from the optical center of the underwater camera to the outer surface of the sealed protective cover glass. The directions of the three coordinate axes are consistent with the inner coordinate system of the sealed protective cover.

[0018] The relative position relationship T2 between the intermediate coordinate system and the outer coordinate system of the sealing protective cover is determined as follows:

[0019] Given the coordinates of the spatial point in the intermediate coordinate system [Xt, Yt, Zt], the coordinates of the spatial point in the outer coordinate system of the sealing protective cover [Xd, Yd, Zd] can be obtained through T2, that is:

[0020]

[0021] The relative position relationship T1 between the inner coordinate system of the sealing protective cover and the camera coordinate system is obtained as follows:

[0022] 4-1) Starting from the optical center O of the underwater camera, draw a perpendicular line to the inner surface of the sealing protective cover. The value of the direction vector corresponding to the perpendicular line in the camera coordinate system is expressed by the column vector vz;

[0023] 4-2) Using the cross product formula Get the inner coordinate system of the sealing protective cover, and the value vy of the Y-axis direction vector in the camera coordinate system;

[0024] 4-3) Calculate the formula by cross product Get the inner coordinate system of the sealing protective cover and the value vx of the X-axis direction vector in the camera coordinate system;

[0025] 4-4) Combine the optical center O of the underwater camera and the distance d from the optical center to the inner surface of the sealed protective cover glass to obtain the rotation and translation transformation matrix T1 required for the three-dimensional coordinates of the spatial point from the inner coordinate system of the sealed protective cover to the camera coordinate system, namely:

[0026]

[0027] According to the underwater camera imaging model, the imaging coordinates corresponding to the given underwater target point D are obtained, specifically:

[0028] S1: Given that the coordinates of the target point D in the intermediate coordinate system are [Xt, Yt, Zt], obtain the coordinates of the spatial point in the outer coordinate system of the protective cover [Xd, Yd, Zd];

[0029] S2: After obtaining the distance m between point A and point B through the binary iteration method, execute step S3;

[0030] Among them, point A represents the intersection of the vertical line of the optical center and the inner surface of the protective cover glass, and point B represents the intersection of the light from the target point D to the optical center position O of the camera and the inner surface of the sealed protective cover glass;

[0031] S3: According to the relationship between the outer coordinate system and the inner coordinate system of the protective cover, the coordinates of point B in the inner coordinate system of the protective cover are obtained as [Xb, Yb, Zb];

[0032] S4: According to the rotation and translation transformation matrix T1 required to transform the three-dimensional coordinates of the space point from the inner coordinate system of the sealing protective cover to the camera coordinate system, the coordinates of point B in the camera coordinate system [Xb′, Yb′, Zb′] are obtained;

[0033] S5: According to the coordinates [Xb′, Yb′, Zb′] of point B in the camera coordinate system, the imaging coordinates corresponding to point B are obtained using the camera imaging model.

[0034] The binary iterative method is specifically:

[0035] S2-1: Define underwater target points and related point parameters:

[0036] Given that the underwater target point is at point D, the light starting from point D passes through point C on the outer surface of the sealed protective cover glass and point B on the inner surface of the sealed protective cover glass in sequence, and finally reaches point O, the optical center of the camera;

[0037] Draw a perpendicular line from point O, the optical center of the camera, to the inner surface of the sealed protective cover, with the foot of the perpendicular being point A. The foot of the perpendicular line from point O to the outer surface of the protective cover glass is point E.

[0038] Then: the distance between point A and point B is defined as m; the distance between point E and point C is defined as q; the distance from point D to line AE is p; the distance from point D to the outer front surface of the protective cover glass is e; the thickness of the glass is h; the distance between the optical center point O and point A is d;

[0039] S2-2: Let the distance from point D to line AE be The distance from point D to the outer front surface of the protective cover glass is e = Zd;

[0040] In the binary iterative calculation process, the lower boundary of m is m1=0, the upper boundary is m2=p, and the initial value of m is: (m1+m2) / 2;

[0041] S2-3: Get the value of m: (m1+m2) / 2;

[0042] S2-4: According to Get the value of the intermediate variable t;

[0043] S2-5: According to the distance m between point A and point B and the value of the intermediate variable t, the distance q between the foot of the perpendicular line of the camera optical center point O on the outer surface of the protective cover glass and the outer surface of the sealed protective cover point C is obtained, that is:

[0044] S2-6: Obtain the calculated distance p' from point D to line AE according to the distance q between the foot E of the perpendicular line from the optical center O to the outer surface of the protective cover glass and point C on the outer surface of the sealed protective cover, that is:

[0045] S2-7: Compare the magnitudes of p' and p. If the deviation between the two is less than the threshold T0, the iteration is completed; if p' > p, let m2 = m, and return to step S2-3; if p' < p, let m1 = m, and return to step S2-3;

[0046] S2-8: Repeat steps S2-3 to S2-7 until the deviation between p' and p is less than the threshold T0 or the number of iterations reaches the upper limit of the allowed number of iterations;

[0047] S2-9: Before executing step S2-3, if p < T0, directly let m = q = 0, and directly execute step S3.

[0048] In the said step S4, obtain the coordinates of point B in the camera coordinate system, specifically:

[0049] Since points O, A, B, E, C, and D are coplanar, and at the same time, the orientations of the coordinate axes of the outer coordinate system of the sealed protective cover and the inner coordinate system of the sealed protective cover are the same. After obtaining the specific value of m through the dichotomy iteration method, the coordinates [Xb, Yb, Zb] of point B in the inner coordinate system of the sealed protective cover can be obtained, that is:

[0050]

[0051] Zb = d

[0052] where [Xd, Yd, Zd] are the coordinates of the underwater target point D in the outer coordinate system of the sealed protective cover.

[0053] A method for calibrating the parameters of an underwater camera imaging model includes the following steps:

[0054] (1) Before assembling the sealed protective cover on the underwater camera, complete the calibration of the internal parameters and distortion coefficients of the camera;

[0055] (2) Fix the underwater camera in an air environment, use a laser tracker to scan the front surface of the sealed protective cover, and establish an intermediate coordinate system;

[0056] (3) Set N non-coplanar space points as calibration reference points, use a laser tracker to measure the coordinates of each calibration reference point in the intermediate coordinate system, denoted as [Xt i , Yt i , Zt i , i = 1,..., N;

[0057] (4) The imaging coordinates of each calibration reference point are extracted by image processing method, denoted as [u i ,v i ],i=1,...,N;

[0058] (5) Refractive index of the sealing cover glass n g Set according to the actual value, set the air refractive index to n a = 1.0, set the refractive index of the external medium of the protective cover n w =1.0;

[0059] (6) Estimate the initial value of the distance d from the optical center of the camera to the inner surface of the sealing protective cover according to the actual relative position relationship between the camera and the sealing protective glass;

[0060] Set the initial value of the Z-axis direction vector of the inner coordinate system of the sealing protective cover in the camera coordinate system to [0,0,1];

[0061] Set the initial value of the relative posture relationship T2 between the intermediate coordinate system and the outer coordinate system of the sealing protective cover to a 4×4 unit matrix;

[0062] (7) According to the underwater camera imaging model, the imaging coordinates of N calibration reference points are obtained, and compared with the imaging point coordinates actually extracted in step (4), and the sum of squares of reprojection errors is obtained. The sum of squares of reprojection errors is used as the objective function, and vz, d, T2 are used as optimization variables. The objective function is iteratively optimized and solved by an iterative optimization algorithm to obtain the optimal solution of the optimization variables vz, d, T2.

[0063] The present invention has the following beneficial effects and advantages:

[0064] 1. The underwater camera imaging model proposed in the present invention takes into account the general situation of camera installation inside the protective cover, allows the camera optical axis and the protective cover glass to be not strictly perpendicular, and also takes into account the thickness of the protective cover glass. It has a wider range of applications and can accurately calculate the imaging coordinates of underwater space points.

[0065] 2. The underwater camera imaging model calibration method proposed in the present invention isolates the calibration environment from the actual use environment, and the calibration process does not need to be performed underwater, which simplifies the cumbersomeness of the previous underwater visual measurement camera calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of the optical path from a spatial point to the optical center of the camera in the protective cover in the present invention;

[0067] Figure 2 It is a schematic diagram of the assembly of the protective cover and the camera in the present invention;

[0068] Figure 3This is a calculation flow chart of the underwater camera imaging model of the present invention;

[0069] Figure 4 This is a flow chart of calibrating imaging model parameters in the present invention. DETAILED DESCRIPTION

[0070] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0071] like Figure 2 As shown, the present invention discloses a schematic diagram of the assembly of a protective cover and a camera of an underwater camera imaging model, which includes: a camera, a sealed protective cover, and a sealed protective cover glass;

[0072] The sealed protective cover glass is a planar structure, and the influence of the glass thickness on the imaging optical path cannot be ignored; the camera observes the underwater target through the sealed protective cover glass in front of the sealed protective cover; when a target point D underwater is given, the three-dimensional coordinates in the intermediate coordinate system associated with the front surface of the sealed protective cover can be obtained through the underwater camera imaging model.

[0073] like Figure 1 As shown, it is a schematic diagram of the optical path from a spatial point to the optical center position of the camera in the protective cover in the present invention. A design method of an underwater camera imaging model in the present invention includes the following steps:

[0074] 1) Determine the refractive index of the sealed protective cover glass, the refractive index of the medium outside the cover, and the thickness of the sealed protective cover glass, that is:

[0075] The front end of the sealed protective cover (i.e. the observation direction of the camera) is agreed to be the refractive index n of the sealed transparent sealed protective cover glass. g , underwater measurement of the refractive index of the ambient water body n w and the refractive index of air n a are known parameters;

[0076] The thickness of the sealing protective cover glass is h, which is a known parameter;

[0077] There is a certain angle θ between the camera optical axis and the normal vector of the sealed protective cover glass. When the angle θ is equal to 0, it means that the camera optical axis is parallel to the normal vector of the sealed protective cover glass.

[0078] The distance from the camera's optical center to the surface of the inner side of the sealed protective cover glass (the side of the protective cover close to the camera) is d;

[0079] 2) Establishing the inner coordinate system and the outer coordinate system of the sealing protective cover;

[0080] In step 2), a coordinate system for the inner side of the sealed protective cover glass is established. The origin of the coordinate system is the foot of a perpendicular line from the optical center of the camera to the inner surface of the sealed protective cover glass. The X-axis is the projection of the X-axis of the camera coordinate system onto the inner surface of the sealed protective cover glass. The Z-axis is the normal vector of the surface of the sealed protective cover glass, pointing to the outside of the protective cover. The Y-axis direction satisfies the right-hand rule.

[0081] Establish a sealed protective cover glass outer side coordinate system. The origin of the coordinate system is the intersection of the extended line of the perpendicular line from the camera optical center to the inner surface of the sealed protective cover glass and the outer surface of the protective cover glass. The directions of the three coordinate axes are consistent with the establishment of the protective cover glass inner side coordinate system.

[0082] 3) Determine the relative position relationship T2 between the intermediate coordinate system and the outer coordinate system of the sealing protective cover;

[0083] 4) Obtain the relative position relationship T1 between the inner coordinate system of the sealing protective cover and the camera coordinate system;

[0084] 5) According to steps 1) to 4), the underwater camera imaging model is established, and the imaging coordinates corresponding to the given underwater target point D are obtained according to the underwater camera imaging model.

[0085] The detailed steps for obtaining the imaging coordinates corresponding to the given underwater target point D are as follows:

[0086] Assuming that the underwater target point is located at point D, the light starting from point D passes through point C on the outer surface of the sealed protective cover glass and point B on the inner surface of the sealed protective cover glass, and finally reaches point O, the optical center.

[0087] The angle between the incident light and the normal vector of the sealed protective cover glass at point C is θ w , the angle between the outgoing light and the normal vector of the sealed protective cover glass is θ g ;

[0088] The angle between the incident light and the normal vector of the sealed protective cover glass at point B is θ g , the angle between the outgoing light and the normal vector of the sealed protective cover glass is θ a ;

[0089] Draw a perpendicular line from the optical center O to the inner surface of the sealed protective cover, with the foot of the perpendicular being point A, and the foot of the perpendicular line from point O to the outer surface of the sealed protective cover glass being point E;

[0090] Then: the distance between point A and point B is defined as m; the distance between point E and point C is defined as q; the distance from point D to line AE is p; the distance from point D to the outer front surface of the protective cover glass is e; the thickness of the glass is h; the distance between the optical center point O and point A is d;

[0091] According to the basic trigonometric function calculation formula (1), we have:

[0092]

[0093] It is known that the refractive indices of the air inside the sealed protective cover, the protective cover glass and the medium outside the protective cover for light are n a 、n g and n w According to Snell's law of refraction, the following equation holds:

[0094] n w ·sin(θ w )=n g ·sin(θ g )=n a ·sin(θ a ) (2)

[0095] Combining formula (1) and formula (2), we have:

[0096]

[0097] The coordinate value of the foot of the perpendicular A in the camera coordinate system can be obtained through the subsequent calibration process. When the coordinates of point A are known, after the target point D is given, in formula (3), m and q are unknowns, t is an intermediate variable, and the variables corresponding to other symbols are all known numbers. As long as m can be calculated, the coordinates of point B in the camera coordinate system can be further calculated, so calculating m becomes the key to solving the underwater camera imaging model. If formula (3) is directly solved analytically, a one-variable multi-time equation will be obtained, and the solution process is also very complicated.

[0098] Considering the above equation, if m is given, t can be calculated, and q can be further obtained. After q is obtained, the guess value p′ of p can be calculated. Under the physical geometric constraints, p′ is a monotonically increasing function of m, and the actual value p of p′ is a known quantity. The maximum value of m will not be greater than p, and the minimum value will not be less than 0. Therefore, the binary search method can be used to iteratively solve m, as follows:

[0099] (1) Take 0 as the lower boundary m1 of m and p as the upper boundary m2 of m;

[0100] (2) Let the initial value of m be (m1+m2) / 2, according to the formula Calculate the value of t;

[0101] (3) According to the values ​​of m and t, the formula Calculate the value of q;

[0102] (4) According to the values ​​of q and t, the formula Calculate the value of p′;

[0103] (5) Compare the magnitudes of p' and p. If the deviation between the two is less than T0 (T0 is a set threshold), the calculation is complete; if p' > p, then let m2 = m; if p' < p, then let m1 = m;

[0104] (6) Repeat steps (3) to (5) until the deviation between p' and p is less than the threshold T0 or the maximum number of iterations is reached;

[0105] (7) Before executing step (1) in the above process, if p < T0, then directly let m = q = 0, and there is no need to enter the subsequent iterative process.

[0106] Since points O, A, B, E, C, and D are coplanar, and at the same time, the orientations of the coordinate axes of the coordinate system outside the protective cover glass are the same as those of the coordinate system inside the protective cover glass. Therefore, after calculating the specific value of m through the bisection iteration method, the coordinates of point B in the coordinate system inside the protective cover glass can be calculated as [Xb, Yb, Zb] using formula (4).

[0107]

[0108] Among them, [Xd, Yd, Zd] are the coordinates of the target point D in the coordinate system outside the sealed protective cover glass.

[0109] Assume that a perpendicular line is drawn from the camera optical center to the inner surface of the sealed protective cover glass. The value of the direction vector corresponding to the perpendicular line in the camera coordinate system is expressed using the column vector vz. At the same time, vz also represents the expression of the direction vector of the Z-axis of the coordinate system inside the protective cover glass in the camera coordinate system. Using formula (5) (where represents the cross product symbol), the value of the direction vector of the Y-axis of the coordinate system inside the protective cover glass in the camera coordinate system, vy, is calculated. Here, it is required that the Y-axis of the coordinate system inside the protective cover glass is perpendicular to both the X-axis of the camera coordinate system and the Z-axis of the coordinate system inside the protective cover glass; then, the value of the direction vector of the X-axis of the coordinate system inside the protective cover glass in the camera coordinate system, vx, is calculated through formula (6). Combining the distance between the optical center O and point A, d, the rotation and translation transformation matrix required for the calculation process of the three-dimensional coordinates of the spatial point from the coordinate system inside the protective cover glass to the camera coordinate system can be obtained. The transformation matrix is shown in formula (7), that is, the step of obtaining the relative pose relationship T1 between the coordinate system inside the protective cover glass and the camera coordinate system is completed, namely:

[0110]

[0111] According to formula (4) and formula (7), the coordinates of point B in the camera coordinate system can be further calculated, as shown in formula (8).

[0112]

[0113] The camera imaging projection model is further used to calculate the imaging coordinates corresponding to point B. The process of using the coordinates of point B in the camera coordinate system to calculate the imaging coordinates is basic knowledge in the field of machine vision measurement and will not be further described here.

[0114] In summary, the underwater camera imaging model can be expressed using formula (9), which is: given the coordinates of point D in the outer coordinate system of the sealed protective cover glass [Xd, Yd, Zd], the imaging coordinates of point D can be calculated using the calculation process F() and configuration parameters.

[0115]

[0116] Where w represents the camera internal parameters and distortion coefficients; n a 、n g and n w Respectively represent the refractive index of the air inside the protective cover, the sealed protective cover glass and the medium outside the protective cover to the light; vz represents the direction vector from the optical center point O and point A; d represents the distance between the optical center point O and point A. In the process of calculating the coordinates of the imaging point, the configuration parameters w, n a 、n g 、n w , vz, and d are all used as known parameters.

[0117] The outer coordinate system of the sealed protective cover glass is established through the camera coordinate system, and the three-dimensional coordinate value of the space point relative to the outer coordinate system of the sealed protective cover glass cannot be directly obtained, but needs to go through an intermediate coordinate system. That is, the coordinates [Xt, Yt, Zt] of the space point in the intermediate coordinate system are directly obtained, and then the coordinates [Xd, Yd, Zd] of the space point in the outer coordinate system of the sealed protective cover glass are calculated through the three-dimensional coordinate transformation method between the intermediate coordinate system and the outer coordinate system of the sealed protective cover glass. Here, the relative position relationship between the intermediate coordinate system and the outer coordinate system of the sealed protective cover glass is also part of the underwater camera imaging model parameters.

[0118] Therefore, for the above step 3), the specific method for determining the relative position relationship T2 between the intermediate coordinate system and the outer coordinate system of the sealing protective cover is as follows:

[0119] First, the method of establishing the intermediate coordinate system is to project the centroid of the front surface of the sealed protective cover onto the outer surface of the sealed protective cover glass, and the projection point is used as the origin of the intermediate coordinate system. The Z axis of the intermediate coordinate system is perpendicular to the outer surface of the protective cover glass and points to the outside of the protective cover. When the operator and the camera inside the protective cover maintain the same observation angle, the X axis of the intermediate coordinate system is horizontal to the right and the Y axis is vertically downward. The three-dimensional coordinate transformation method of the spatial point between the intermediate coordinate system and the coordinate system outside the protective cover glass is shown in formula (10), where [Xt, Yt, Zt] is the coordinate of the spatial point in the intermediate coordinate system.

[0120]

[0121] Formula (9) is adjusted to meet general usage requirements, as shown in formula (11), where H() represents the calculation process of converting the coordinates in the temporary coordinate system into the coordinates in the outer side coordinate system of the protective cover glass, and the calculation process is shown in formula (10).

[0122]

[0123] Before assembling the camera inside the protective cover, the configuration parameters w and n need to be calibrated using the Zhang Zhengyou calibration method. a 、n g and n w Here it is used directly as a valued parameter.

[0124] like Figure 4 As shown, the calibration method of the present invention is to calibrate the configuration parameters vz, d and T2 that characterize the relative position relationship between the camera and the sealed protective cover glass inside the protective cover. The calibration method used to achieve the above purpose is:

[0125] The calibration process is carried out in the air, so the refractive index of the medium outside the protective cover is equal to the refractive index of air during the calibration process. The specific camera imaging model calibration method is as follows:

[0126] (1) Before the underwater camera is equipped with a sealed protective cover, the camera's internal parameters and distortion coefficients are calibrated;

[0127] (2) Fix the underwater camera in an air environment, use a laser tracker to scan the front surface of the sealed protective cover, and establish an intermediate coordinate system;

[0128] (3) Set N non-coplanar spatial points as calibration reference points, and use a laser tracker to measure the coordinates of each calibration reference point in the intermediate coordinate system, denoted as [Xt i ,Yt i ,Zt i ],i=1,...,N;

[0129] (4) The imaging coordinates of each calibration reference point are extracted by image processing method, denoted as [u i ,v i ],i=1,...,N;

[0130] (5) Refractive index of the sealing cover glass n g Set according to the actual value, set the air refractive index to n a = 1.0, set the refractive index of the external medium of the protective cover n w =1.0;

[0131] (6) Estimate the initial value of the distance d from the optical center of the camera to the inner surface of the sealing protective cover according to the actual relative position relationship between the camera and the sealing protective glass;

[0132] Set the initial value of the Z-axis direction vector of the inner coordinate system of the sealing protective cover in the camera coordinate system to [0,0,1];

[0133] Set the initial value of the relative posture relationship T2 between the intermediate coordinate system and the outer coordinate system of the sealing protective cover to a 4×4 unit matrix;

[0134] (7) According to the underwater camera imaging model, the imaging coordinates of N calibration reference points are obtained according to formula (12), and compared with the actual imaging point coordinates extracted in step (4), and the cumulative sum of squares of the reprojection errors of the N imaging points is calculated according to formula (13). The cumulative sum of squares of the reprojection errors is used as the objective function, vz, d, T2 are used as optimization variables, and an iterative optimization algorithm is used. With vz, d, T2 as optimization variables, the objective function represented by formula (13) is iteratively optimized and solved, that is:

[0135]

[0136] Embodiment 1:

[0137] like Figure 3 As shown in FIG. 1 , it is a flow chart of underwater camera imaging model calculation in this embodiment. Under the premise that the parameters of underwater camera imaging model are known, after the coordinates of a space point in the intermediate coordinate system are given, the imaging coordinates corresponding to the space point can be calculated using the model. Figure 1 As shown, the specific implementation steps are:

[0138] 1) The coordinates of the spatial point D in the intermediate coordinate system are known to be [Xt, Yt, Zt], and the coordinates of the spatial point in the outer coordinate system of the protective cover glass [Xd, Yd, Zd] are calculated by formula (10);

[0139] 2) Order e=Zd;

[0140] 3) Let the lower boundary of the value of m be m1 = 0, the upper boundary be m2 = p, and the initial value of m be (m1 + m2) / 2. Calculate the value of t according to the formula ;

[0141] 4) Calculate the value of q according to the values of m and t according to the formula ;

[0142] 5) Calculate the value of p′ according to the values of q and t according to the formula ;

[0143] 6) Compare the magnitudes of p′ and p. If the deviation between the two is less than T0 (T0 = 1E - 8), the calculation is completed; if p′ > p, then let m2 = m; if p′ < p, then let m1 = m;

[0144] 7) Repeat steps 2) to 5) until the deviation between p′ and p is less than the threshold T0 or the number of iterations reaches 30 times;

[0145] 8) Before executing step 3) in the above process, judge that if p < T0, then directly let m = q = 0, and there is no need to enter the subsequent iteration process;

[0146] 9) Calculate the coordinates of point B in the coordinate system inside the protective cover glass as [Xb, Yb, Zb] according to formula (4);

[0147] 10) Calculate the coordinates [Xb′, Yb′, Zb′] of point B in the camera coordinate system according to formula (8);

[0148] 11) Calculate the imaging coordinates corresponding to point B using the camera imaging projection model.

[0149] When using the above underwater imaging model to calculate the imaging coordinates of spatial points, taking K9 glass as an example for the protective cover glass, set n g = 1.51630, set the refractive index of air n a = 1.0. If it is measured in fresh water at 16°C, set the refractive index of the medium outside the protective cover n w = 1.3333. If it is measured in sea water at 16°C, set the refractive index of the medium outside the protective cover n w = 1.3415. In addition, the refractive index of the underwater environment will change due to factors such as impurities, salinity, and temperature. The refractive indices of fresh water and sea water given here are for reference only. When implementing specific measurement work, the actual refractive index of the underwater environment at that time should be used as the standard.

[0150] As Figure 4 shown, the flowchart of calibrating the imaging model parameters in this embodiment. The implementation steps of a method for calibrating the parameters of an underwater camera imaging model in this embodiment are as follows:

[0151] 1) Before the camera is equipped with a protective cover, the Zhang Zhengyou calibration method is used to calibrate the camera's internal parameters and distortion coefficients;

[0152] 2) Fix the camera equipped with the protective cover, use a laser tracker to scan the front surface of the protective cover glass, establish the intermediate coordinate system OXY plane, scan the front face contour of the protective cover to establish the centroid position, and use the OXY plane projection of the centroid position of the front face contour of the protective cover as the origin position of the intermediate coordinate system. Use the normal vector of the front surface of the protective cover glass to establish the intermediate coordinate system Z axis, with the positive direction of the Z axis pointing to the outside of the protective cover glass. Use the horizontal right feature of the protective cover to establish the intermediate coordinate system X axis, and the Y axis satisfies the right-hand rule;

[0153] 3) In this embodiment, 30 non-coplanar spatial points are set as calibration reference points, and the coordinates of each calibration reference point in the intermediate coordinate system are measured using a laser tracker, which is recorded as [Xt i ,Yt i ,Zt i ],i=1,...,30;

[0154] 4) Extract the imaging coordinates of each calibration reference point through image processing, denoted as [u i ,v i ],i=1,...,30;

[0155] 5) The protective cover glass of this embodiment uses K9 glass as an example, and sets n g =1.51630, set the air refractive index n a =1.0, because the calibration is done in air, the refractive index of the medium outside the protective cover is set to n w =1.0;

[0156] 6) Set the initial value of d to 10, the initial value of vz to [0,0,1], and the initial value of T2 to 4×4 unit matrix;

[0157] 7) Taking vz, d, T2 in formula (12) as the optimization variables, the objective function represented by formula (13) is solved by the LM iterative optimization algorithm to obtain the optimal solution of the optimization variables vz, d, T2.

[0158] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0159] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An underwater camera imaging model, characterized in that: include: Camera, sealed protective cover, sealed protective cover glass; The sealed protective cover glass is a flat structure, and the effect of the glass thickness on the imaging optical path cannot be ignored; The camera observes underwater targets through the sealed protective cover glass in front of the sealed protective cover; when a target point D underwater is given, the three-dimensional coordinates in the intermediate coordinate system associated with the front surface of the sealed protective cover can be obtained through the underwater camera imaging model.

2. An underwater camera imaging model according to claim 1, characterized in that: The intermediate coordinate system is as follows: the centroid of the front surface of the sealed protective cover is used to project onto the outer surface of the sealed protective cover glass, the projection point is used as the origin of the intermediate coordinate system, the Z axis of the intermediate coordinate system is perpendicular to the outer surface of the sealed protective cover glass, and points to the outside of the sealed protective cover; when the operator who establishes the coordinate system maintains the same observation angle as the camera inside the sealed protective cover, the X axis of the intermediate coordinate system is horizontal to the right, and the Y axis is vertically downward.

3. The method for designing an underwater camera imaging model according to claim 1, characterized in that: The following steps are involved: 1) Determine the refractive index of the sealed protective cover glass, the refractive index of the medium outside the cover, and the thickness of the sealed protective cover glass; 2) Establishing the inner coordinate system and the outer coordinate system of the sealing protective cover; 3) Determine the relative position relationship T2 between the intermediate coordinate system and the outer coordinate system of the sealing protective cover; 4) Obtain the relative position relationship T1 between the inner coordinate system of the sealing protective cover and the camera coordinate system; 5) According to steps 1) to 4), the underwater camera imaging model is established, and the imaging coordinates corresponding to the given underwater target point D are obtained according to the underwater camera imaging model.

4. The method for designing an underwater camera imaging model according to claim 3, characterized in that: The establishment of the inner coordinate system of the protective cover and the outer coordinate system of the protective cover is specifically as follows: 2-1) Establishing the inner coordinate system of the protective cover: The origin of the coordinate system is the foot of a perpendicular line drawn from the optical center of the underwater camera to the inner surface of the sealed protective cover glass. The X-axis is the projection of the X-axis of the camera coordinate system onto the inner surface of the sealed protective cover glass. The Z-axis is the normal vector of the sealed protective cover surface, pointing to the outside of the sealed protective cover. The Y-axis direction satisfies the right-hand rule. 2-2) Establishing the outer coordinate system of the sealing protective cover: The origin of the coordinate system is the foot of a perpendicular line drawn from the optical center of the underwater camera to the outer surface of the sealed protective cover glass. The directions of the three coordinate axes are consistent with the inner coordinate system of the sealed protective cover.

5. The method for designing an underwater camera imaging model according to claim 3, characterized in that: The relative position relationship T2 between the intermediate coordinate system and the outer coordinate system of the sealing protective cover is determined as follows: Given the coordinates of the spatial point in the intermediate coordinate system [Xt, Yt, Zt], the coordinates of the spatial point in the outer coordinate system of the sealing protective cover [Xd, Yd, Zd] can be obtained through T2, that is:

6. The method for designing an underwater camera imaging model according to claim 3, characterized in that: The relative position relationship T1 between the inner coordinate system of the sealing protective cover and the camera coordinate system is obtained as follows: 4-1) starting from the optical center O of the underwater camera, a perpendicular line is drawn to the inner surface of the sealing protective cover, and the value of the direction vector corresponding to the perpendicular line in the camera coordinate system is expressed by a column vector vz; 4-2) Using the cross product formula Get the inner coordinate system of the sealing protective cover, and the value vy of the Y-axis direction vector in the camera coordinate system; 4-3) Calculate the formula by cross product Get the inner coordinate system of the sealing protective cover and the value vx of the X-axis direction vector in the camera coordinate system; 4-4) Combine the optical center O of the underwater camera and the distance d from the optical center to the inner surface of the sealed protective cover glass to obtain the rotation and translation transformation matrix T1 required for the three-dimensional coordinates of the spatial point from the inner coordinate system of the sealed protective cover to the camera coordinate system, namely:

7. The method for designing an underwater camera imaging model according to claim 3, characterized in that: According to the underwater camera imaging model, the imaging coordinates corresponding to the given underwater target point D are obtained, specifically: S1: Given that the coordinates of the target point D in the intermediate coordinate system are [Xt, Yt, Zt], obtain the coordinates [Xd, Yd, Zd] of this spatial point in the outer coordinate system of the protective cover. S2: After obtaining the distance m between point A and point B through the bisection iteration method, execute step S3. Among them, point A represents the intersection point of the perpendicular line of the optical center and the inner surface of the protective cover glass, and point B represents the intersection point of the light ray starting from the target point D and reaching the camera optical center position O and the inner surface of the sealed protective cover glass. S3: According to the relationship between the outer coordinate system and the inner coordinate system of the protective cover, obtain the coordinates [Xb, Yb, Zb] of point B in the inner coordinate system of the protective cover. S4: According to the rotation and translation transformation matrix T1 required for the transformation of the three-dimensional coordinates of the spatial point from the inner coordinate system of the sealed protective cover to the camera coordinate system, obtain the coordinates [Xb′, Yb′, Zb′] of point B in the camera coordinate system. S5: According to the coordinates [Xb′, Yb′, Zb′] of point B in the camera coordinate system, use the camera imaging model to obtain the imaging coordinates corresponding to point B.

8. The method for designing an underwater camera imaging model according to claim 7, characterized in that: The bisection iteration method is specifically as follows: S2-1: Define the underwater target point and related point position parameters: Given that the position of the underwater target point is point D, the light ray starting from point D passes through point C on the outer surface of the sealed protective cover glass, point B on the inner surface of the sealed protective cover glass, and finally reaches the camera optical center position O. Draw a perpendicular line from the camera optical center position O to the inner surface of the protective cover, and the foot of the perpendicular is point A. The foot of the perpendicular of point O on the perpendicular line of the outer surface of the protective cover glass is point E. Then: The distance between point A and point B is defined as m; the distance between point E and point C is defined as q; the distance from point D to the straight line AE is p; the distance from point D to the outer front surface of the protective cover glass is e; the thickness of the glass is h; the distance between the optical center O and point A is d. S2-2: Let the distance from point D to line AE be The distance from point D to the outer front surface of the protective cover glass is e = Zd; Let the lower boundary m1 = 0 and the upper boundary m2 = p of the value of m in the bisection iteration calculation process, and the initial value of m is: (m1 + m2) / 2. S2-3: Obtain the value of m as: (m1 + m2) / 2. S2-4: According to Get the value of the intermediate variable t; S2-5: According to the distance m between point A and point B and the value of the intermediate variable t, the distance q between the foot of the perpendicular line of the camera optical center point O on the outer surface of the protective cover glass and the outer surface of the sealed protective cover point C is obtained, that is: S2-6: According to the distance q between the foot of the perpendicular line E of the optical center point O on the outer surface of the protective cover glass and the point C on the outer surface of the sealed protective cover, obtain the value of the calculated distance p′ from point D to the straight line AE, that is: S2-7: Compare the magnitudes of p′ and p. If the deviation between the two is less than the threshold T0, the iteration is completed; if p′ > p, then let m2 = m and return to step S2-3; if p′ < p, then let m1 = m and return to step S2-3. S2-8: Repeat steps S2-3 to S2-7 until the deviation between p′ and p is less than the threshold T0 or the number of iterations reaches the upper limit of the allowable number of iterations. S2-9: Before executing step S2-3, if p < T0, directly let m = q = 0 and directly execute step S3.

9. The method for designing an underwater camera imaging model according to claim 7, characterized in that: In step S4, obtaining the coordinates of point B in the camera coordinate system is specifically as follows: Since points O, A, B, E, C, and D are coplanar, and at the same time, the orientations of the respective coordinate axes of the outer coordinate system of the sealed protective cover and the inner coordinate system of the sealed protective cover are the same. After obtaining the specific value of m through the bisection iteration method, the coordinates [Xb, Yb, Zb] of point B in the inner coordinate system of the sealed protective cover can be obtained, that is: Zb = d Among them, [Xd, Yd, Zd] is the coordinate of the underwater target point D in the outer coordinate system of the sealing protective cover.

10. The method for calibrating parameters of an underwater camera imaging model according to claim 1, characterized in that: The following steps are involved: (1) Before the underwater camera is equipped with a sealed protective cover, the camera's internal parameters and distortion coefficients are calibrated; (2) Fix the underwater camera in an air environment, use a laser tracker to scan the front surface of the sealed protective cover, and establish an intermediate coordinate system; (3) Set N non-coplanar spatial points as calibration reference points, and use a laser tracker to measure the coordinates of each calibration reference point in the intermediate coordinate system, denoted as [Xt i ,Yt i ,Zt i ],i=1,...,N; (4) The imaging coordinates of each calibration reference point are extracted by image processing method, denoted as [u i ,v i ],i=1,...,N; (5) Refractive index of sealing cover glass n g Set according to the actual value, set the air refractive index to n a = 1.0, set the refractive index of the external medium of the protective cover n w =1.0; (6) Estimate the initial value of the distance d from the optical center of the camera to the inner surface of the sealing protective cover according to the actual relative position relationship between the camera and the sealing protective glass; Set the initial value of the Z-axis direction vector of the inner coordinate system of the sealing protective cover in the camera coordinate system to [0,0,1]; Set the initial value of the relative posture relationship T2 between the intermediate coordinate system and the outer coordinate system of the sealing protective cover to a 4×4 unit matrix; (7) According to the underwater camera imaging model, the imaging coordinates of N calibration reference points are obtained, and compared with the imaging point coordinates actually extracted in step (4), and the sum of squares of reprojection errors is obtained. The sum of squares of reprojection errors is used as the objective function, and vz, d, T2 are used as optimization variables. The objective function is iteratively optimized and solved by an iterative optimization algorithm to obtain the optimal solution of the optimization variables vz, d, T2.

Images