Underwater pile foundation disease three-dimensional reconstruction method

Abstract

The invention relates to the technical field of bridge pile foundation concrete, in particular to an underwater pile foundation disease three-dimensional reconstruction method, which comprises the following steps: establishing according to a human binocular vision system bionics principle, based on a parallax theory, using two cameras at different positions to shoot the same target feature point in a scene at the same time, and the three-dimensional information of the measured target object is obtained by calculating the position deviation between the corresponding image points of the two images through a triangular formula. According to the method, the position and posture relation of the two cameras can be calibrated, the image planes of the two cameras are converted into the position states under the ideal condition, calculation of the three-dimensional coordinates of the pile foundation disease feature points is achieved, and therefore three-dimensional reconstruction of the disease is conducted.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge pile foundation concrete, and particularly to a three-dimensional reconstruction method for underwater pile foundation diseases. Background Art

[0002] Underwater pile foundation concrete is prone to diseases due to factors such as water flow scouring, seawater corrosion, ship collision, and wet-dry cycles. If not detected in time, these diseases will affect the bearing capacity of the upper structure and the stability of the structure to a certain extent, and will seriously affect the structural safety and cause major safety accidents when severe.

[0003] Currently, although some experts at home and abroad have conducted research on underwater pile foundation disease detection, the mainstream disease imaging technology is still that the detector dives to explore or uses a monocular camera to take disease imaging for disease image detection, which not only has a high cost, but also the detection effect needs to be improved.

[0004] The traditional underwater pile foundation concrete detection technology uses divers to manually measure surface diseases, which is not only inefficient but also has a high risk factor. The underwater environment is dim, and using an underwater camera to take images is not only blurry but also has a deviation in the imaging position. Therefore, the present application provides a three-dimensional reconstruction technology for underwater pile foundation diseases to solve the problem of imaging position deviation to a certain extent. Summary of the Invention

[0005] The purpose of the present invention is to solve the deficiencies in the prior art and propose a three-dimensional reconstruction method for underwater pile foundation diseases. This method can calibrate the pose relationship between two cameras, transform the image planes of the two cameras into an ideal position state, calculate the three-dimensional coordinates of the characteristic points of the pile foundation diseases, and thus perform three-dimensional reconstruction of the diseases.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions:

[0007] A three-dimensional reconstruction method for underwater pile foundation diseases includes the following steps:

[0008] Step 1: Based on the parallax theory, use two cameras at different positions to simultaneously shoot the same target characteristic point in the scene. By calculating the position deviation between the corresponding image points of the two images and using the trigonometric formula, obtain the three-dimensional information of the measured target object;

[0009] Step 2: According to the different placements of the cameras, the measurement model is divided into two imaging measurement models: ideal and non-ideal. In actual calculations, usually calibrate the pose relationship between the two cameras, transform the image planes of the two cameras into an ideal position state, and calculate the three-dimensional coordinates of the characteristic points;

[0010] Step 3: Adopt the form of four-dimensional light parameterization to facilitate the recording and tracking of the direction information and position information of light during underwater propagation, and realize the modeling and analysis of the underwater camera imaging process;

[0011] Step 4: Combine the four-dimensional light parameterization representation method to establish an underwater camera multi-layer plane refraction imaging model;

[0012] Step 5: Establish an underwater measurement model;

[0013] Step 6: Based on the multi-layer refraction imaging model, obtain the three-dimensional information of the measured object point.

[0014] Preferably, in Step 2, the specific method is as follows:

[0015] Assume that O 1 and O 2 are the optical center points of the left and right cameras respectively, and the baseline distance between them is T. Any object point P 0 in space is projected onto the left and right image planes as P l and P r respectively. Among them, x l and x r are the abscissas of the image points on the image plane, then the parallax between the projection points can be obtained as d = x l -x r ;

[0016] From the geometric relationship of the spatial triangle, the following formula can be obtained:

[0017]

[0018] x l is the abscissa of the left image point on the plane, x r is the abscissa of the right image point on the plane, T is the baseline distance between them, f is the focal length, and Z is the vertical distance from the object point to the camera plane;

[0019] After arranging the above formula, the distance from the object point to the baseline bracket can be found as:

[0020]

[0021] x l is the abscissa of the left image point on the plane, x r is the abscissa of the right image point on the plane, T is the baseline distance between them, f is the focal length, and Z is the vertical distance from the object point to the camera plane;

[0022] From Equation (2), when the parallax value of the image and the pose relationship between the cameras are obtained, the coordinate of the spatial object point P in the Z direction can be obtained; similarly, according to the triangle similarity relationship, the coordinates in the X and Y directions can be obtained, so as to obtain the three-dimensional information of the object point.

[0023] Preferably, in step 3, the specific method is as follows:

[0024] Light ray oo d vertically passes through two parallel planes with a distance of 1 unit length and intersects the two planes at points O and O respectively d ; among them, the matrix [u, v] T is used to represent the position point O of the light ray to record the position information of the light ray, and the matrix [s, t] T is used to represent the direction vector of the light ray to record the direction information of the light ray; in addition, for any light ray in the scene, it can also be represented by the direction vector a and the position q of this point. The relationship between the two representation methods is as follows:

[0025]

[0026] L is the unit length representation method, a is the direction vector, and q is the position of the object point.

[0027] Preferably, in step 4, the specific method is as follows:

[0028] The camera coordinate system is located at the camera optical center, its z-axis is parallel to the camera optical axis, n is the normal of the multi-layer plane and is perpendicular to the interface. Taking the normal n of the multi-layer interface as the z-axis and the cross product of the normal n and the camera z-axis as the x-axis, a multi-layer plane refraction coordinate system is constructed. The conversion relationship between the two coordinate systems is as follows:

[0029]

[0030] Among them, P c is the three-dimensional point coordinate in the camera coordinate system, P r is the three-dimensional point coordinate in the multi-layer refraction coordinate system, n c is the interface normal vector in the camera coordinate system, c R r is the rotation matrix of the multi-layer refraction coordinate system relative to the camera coordinate system and c t r is the translation matrix of the multi-layer refraction coordinate system relative to the camera coordinate system; the conversion relationship between the object coordinate system and the multi-layer refraction coordinate system is:

[0031] P r = r R o P o + r t o (5)

[0032] P r is the three-dimensional point coordinate in the multi-layer refraction coordinate system, r R 0is the rotation matrix of point O in the multi-layer refraction coordinate system relative to the camera coordinate system, and P 0 is the three-dimensional point coordinate of point O in the multi-layer refraction coordinate system, r t 0 is the translation matrix of point O in the multi-layer refraction coordinate system relative to the camera coordinate system;

[0033] The position plane [u, v] of the four-dimensional parameter ray is defined on the xy plane of the multi-layer plane refraction coordinate system and coincides with its coordinate origin. The direction plane [s, t] is one unit length away from the position plane and is parallel to the position plane; where d is the distance from the camera optical center to the interface, and d i is the distance between interfaces, and u i is the refractive index of each layer of medium.

[0034] Preferably, in step 4, the underwater camera imaging process based on the multi-layer plane refraction model is as follows: The ray corresponding to the object point P passes through m layers of medium refraction, propagates backward to the camera optical center, and intersects the imaging plane at point P w (i.e., the underwater image point). In an actual underwater imaging system, it only includes three layers of medium: water, glass, and air. Since the glass thickness d 1 is much smaller than the propagation distance d of the ray in water 2 , the ray only undergoes a slight radial shift and its direction remains unchanged during this propagation process. Therefore, in actual calculations, the glass layer is generally ignored. According to the principle of reversibility of light paths, the underwater camera imaging process is analyzed using the reverse analysis method:

[0035] (1) According to the principle of pinhole imaging, the relationship between the object point P c (x, y, z) and the image point m = [u o , v o in the camera coordinate system can be established as: T In the formula, K is the internal parameter matrix of the camera, and any pixel

[0036]

[0037] determines a ray passing through the optical center and the pixel; determines a ray passing through the optical center and the pixel;

[0038] determines a ray passing through the optical center and the pixel;

[0039] (2) In the camera coordinate system, the ray direction I c can be expressed as:

[0040]

[0041] (3) According to the conversion relationship between coordinate systems in formula (4), the ray direction I r can be expressed as:

[0042]

[0043] (4) The four-dimensional parametric ray can be described as:

[0044]

[0045] (5) After the ray propagates a certain distance d on the xy plane of the multi-layer refraction coordinate system, the expression is as follows:

[0046]

[0047] In the formula, T(d) is the mathematical expression after the ray propagates the distance d, represents the Kronecker product;

[0048] (6) When the ray undergoes refraction, according to the law of refraction, the refraction process of the ray can be expressed as:

[0049]

[0050] Among them, μ and μ′ are the refractive indices of the incident medium and the emerging medium respectively; R[s, t, μ, μ′] is the refraction expression; α[s, t, μ, μ′] and β[s, t, μ, μ′] are the undetermined parameters of the change in the ray direction after refraction. The specific solution process is as follows: Assume that the expression of the incident ray is L(u, v, s, t) T , and the expression of the refracted ray is L(u′, v′, s′, t′) T , then the angle of incidence of the ray can be expressed as:

[0051]

[0052] Similarly, the angle of refraction of the ray can be expressed as According to the law of refraction μsin(θ) = μ′sin(θ′), By combining, we can get:

[0053]

[0054] Since the refracted ray and the incident ray are in the same plane, the constraint s′t = st′ is satisfied. Therefore,

[0055] In summary, we can get:

[0056]

[0057] (7) Ray L r propagates the distance d 0 , and enters the medium μ 0 from the medium μ 1 and then, the ray can be expressed as:

[0058]

[0059] For light i L r the propagation distance d i and then enters the medium μ i from the medium μ i+1 After refraction, the light ray can be expressed as:

[0060]

[0061] (8) When the light passes through multiple layers of media, the refracted light ray can be expressed as:

[0062]

[0063] Preferably, in step 5, the specific method is as follows:

[0064] Assume that the focal lengths of the left and right cameras of the model are f, and the baseline distance is d 0 , the line connecting the optical centers between the two cameras is parallel to the surface of the optical glass window and also parallel to the set coordinate axis; the distance from the optical center to the waterproof glass is g, and the glass thickness is h. Among them, the refractive indices of air, glass, and seawater are n 0 , n 1 and n 2 , the three-dimensional coordinates of the underwater object point in the left camera coordinate system are P, and its pixel coordinates in the left and right cameras are q l and q r ;

[0065] Assume that the four-dimensional parameter light ray propagates a distance d on the plane perpendicular to the multi-layer refraction coordinate system. At this time, the expression of the light ray is:

[0066]

[0067] When the light ray refracts underwater, assume that the refractive index of the incident light medium is n 1 , and the refractive index of the refracted light medium is n 2 , the incident angle is θ 1 , and the refraction angle is θ 2 , then according to the law of refraction:

[0068]

[0069]

[0070]

[0071] Since the refracted light ray and the normal of the incident light ray are in the same plane, the following constraints are satisfied:

[0072] s′t = s′t (21)

[0073]

[0074] Subsequently, the direction of the light ray can be obtained as follows:

[0075]

[0076] In summary, the propagation process of the refraction of the four-dimensional parameter light ray can be represented by a matrix as follows:

[0077]

[0078] The connection line between an image point of the camera and the optical center determines a light ray. The left camera point and q l = [x 1 , y 1 , the right camera point q r = [x 2 , y 2 , and the principal point position is q 0 = [x 0 , y 0 ; where the coordinate unit is mm. If it is in pixels, the conversion can be done first as [x 1 , y 1 = pix_width[x pix , y pix . The light rays determined by the left and right camera image points can be described by the following formulas respectively:

[0079]

[0080] When the system parameters are known, the object point coordinates can be obtained by finding the intersection of the two light rays. Further, for the convenience of research, the principle of reversibility of the optical path is continued to analyze the underwater imaging process. Taking the left camera as an example, the process of the light ray transmitting from a certain pixel point on the image plane to the object point in water is as follows:

[0081] (1) The light ray travels a distance g in the air and reaches the inner surface of the glass:

[0082]

[0083] (2) The light ray enters the glass from the air and undergoes refraction:

[0084]

[0085] (3) The light ray travels a distance h in the glass and reaches the outer surface of the glass:

[0086]

[0087] (4) The light ray refracts at the surface of the glass, from the glass medium to the seawater medium:

[0088]

[0089] (5) It propagates an unknown distance depth in the seawater and reaches the object point:

[0090]

[0091] In summary, the light ray propagation process from the pixel light ray to the object point position is expressed as follows:

[0092]

[0093] Similarly, for the right camera, we can get:

[0094]

[0095] Since equations (31) and (32) represent the same object point, therefore, by solving the equations simultaneously, we can obtain the object point distance (the two straight lines intersect at the depth depth), and it satisfies the following constraint equations:

[0096]

[0097] where, u 1 , v 1 , s 1 , t 1 , u 2 , v 2 , s 2 and t 2 are the outgoing light rays from the glass to the seawater surface.

[0098] Preferably, in step 6, the specific method is as follows:

[0099] For any point P in space, the corresponding pixels on the left and right camera planes are l m and r m , respectively calculate the four-dimensional light rays l m and r m corresponding to the last layer of medium after refraction through multiple interfaces and and convert them into the corresponding points and vectors l a , lq, r a and r q ; then, convert the points and vectors r q , r q in the right camera coordinate system to the left camera coordinate system, r a′ = l R r × r a, rq′ = l R r × r q + l t r The point P in the left camera coordinate system is simultaneously located on the light rays and ; thus, the following two constraints are satisfied:

[0100]

[0101] In Equation (34), the symbol × represents the vector cross product. By using an anti-symmetric matrix to replace the cross product, the following equation can be obtained:

[0102]

[0103] Performing singular value decomposition on Equation (35) can solve for the three-dimensional coordinates of the object point P in the left camera coordinate system, that is, obtaining the three-dimensional information of the measured object point.

[0104] By adopting the above technical solution: using multiple cameras to take pictures, calibrating and correcting the images, restoring the accurate position of the disease and performing three-dimensional reconstruction, providing accurate original data for disease image recognition.

[0105] Compared with the prior art, the present invention has the following beneficial effects:

[0106] While satisfying the acquisition of clear images, the present invention uses two cameras at different positions to simultaneously capture the same target feature point in the scene. By calculating the position deviation between the corresponding image points of the two images and using the trigonometric formula, the three-dimensional information of the measured target object is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] Figure 1 is the measurement principle diagram of the present invention;

[0108] Figure 2 is the ideal model diagram of three-dimensional measurement in the present invention;

[0109] Figure 3 is the four-dimensional parameterization representation diagram of the light rays in the present invention;

[0110] Figure 4 is the multi-layer plane refraction imaging model diagram of the present invention;

[0111] Figure 5 is the underwater measurement model diagram of the present invention;

[0112] Figure 6 is the underwater imaging model diagram of the present invention;

[0113] Figure 7This is a comparison chart for detecting concrete diseases of underwater pile foundations in the embodiments of the present invention; Figure (a) is a comparison chart for measuring the damaged area of concrete; Figure (b) is a comparison chart for measuring the exposed reinforcement length of concrete. Detailed implementation manners

[0114] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, so that those skilled in the art can better understand the advantages and features of the present invention, and thus more clearly define the protection scope of the present invention. The embodiments described herein are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0115] A three-dimensional reconstruction method for underwater pile foundation diseases includes the following steps:

[0116] Step 1: Based on the bionics principle of the human binocular vision system, and based on the parallax theory, two cameras at different positions are used to simultaneously capture the same target feature point in the scene. By calculating the position deviation between the corresponding image points of the two images, and using the trigonometric formula to obtain the three-dimensional information of the measured target object, the measurement principle is as Figure 1 shown.

[0117] Step 2: According to the different placements of the cameras, the measurement model is divided into two imaging measurement models: ideal and non-ideal. In actual calculations, usually, the pose relationship between the two cameras is calibrated, and the image planes of the two cameras are transformed into the position states in the ideal situation to realize the calculation of the three-dimensional coordinates of the feature points.

[0118] Step 3: The process of light propagation and imaging underwater is regarded as the process of light propagating in multiple parallel media, and refraction occurs at different interfaces. The refracted light is finally projected onto the image plane of the camera. Here, the four-dimensional parameterization form of light is adopted to conveniently record and track the direction information and position information of the light during the underwater propagation process, so as to realize the modeling and analysis of the underwater camera imaging process.

[0119] Step 4: Combine the four-dimensional parameterization representation method of light to establish an underwater camera multi-layer plane refraction imaging model;

[0120] Step 5: Establish an underwater measurement model;

[0121] Step 6: Based on the multi-layer refraction imaging model, obtain the three-dimensional information of the measured object point.

[0122] Specifically, in step 2, the ideal model of three-dimensional measurement is as Figure 2 shown, and the specific method is as follows:

[0123] Assume O 1and O 2 are the optical center points of the left and right cameras respectively, and the baseline distance between the two is T. Any object point P in space 0 The projection points on the left and right image planes are P l and P r respectively, where x l and x r are the abscissas of the image points on the image plane. Then the parallax between the projection points can be obtained as d = x l - x r ;

[0124] From the geometric relationship of the spatial triangle, the following formula can be obtained:

[0125]

[0126] x l is the abscissa of the left image point on the plane, x r is the abscissa of the right image point on the plane, T is the baseline distance between the two, f is the focal length, and Z is the vertical distance from the object point to the camera plane;

[0127] After arranging the above formula, the distance from the object point to the baseline support can be found as:

[0128]

[0129] x l is the abscissa of the left image point on the plane, x r is the abscissa of the right image point on the plane, T is the baseline distance between the two, f is the focal length, and Z is the vertical distance from the object point to the camera plane;

[0130] From Equation (2), when the parallax value of the image and the pose relationship between the cameras are obtained, the coordinate of the spatial object point P in the Z direction can be obtained; similarly, the coordinates in the X and Y directions can be obtained according to the triangle similarity relationship, so that the three-dimensional information of the object point can be obtained.

[0131] Preferably, in step 3, as Figure 3 shown, the specific method is as follows:

[0132] The light oo d vertically passes through two parallel planes with a distance of 1 unit length and intersects the two planes at points o and O d respectively; among them, the matrix [u, v] T is used to represent the position point O of the light ray to record the position information of the light ray, and the matrix [s, t] T is used to represent the direction vector of the light ray to record the direction information of the light ray; in addition, for any light ray in the scene, it can also be represented by the direction vector a and the position q of this point. The relationship between the two representation methods is as follows:

[0133]

[0134] L represents the unit length representation method, a is the direction vector, and q is the position of the object point.

[0135] Preferably, in step 4, as Figure 4 shown, the specific method is as follows:

[0136] The camera coordinate system is located at the camera optical center, its z-axis is parallel to the camera optical axis, n is the normal of the multi-layer plane, and is perpendicular to the interface. Taking the multi-layer interface normal n as the z-axis and the cross product of the normal n and the camera z-axis as the x-axis, a multi-layer plane refraction coordinate system is constructed. The conversion relationship between the two coordinate systems is as follows:

[0137]

[0138] Among them, P c is the three-dimensional point coordinate in the camera coordinate system, P r is the three-dimensional point coordinate in the multi-layer refraction coordinate system, n c is the interface normal vector in the camera coordinate system, c R r is the rotation matrix of the multi-layer refraction coordinate system relative to the camera coordinate system and c t r is the translation matrix of the multi-layer refraction coordinate system relative to the camera coordinate system; the conversion relationship between the object coordinate system and the multi-layer refraction coordinate system is:

[0139] p r = r R o P o + r t o (5)

[0140] P r is the three-dimensional point coordinate in the multi-layer refraction coordinate system, r R 0 is the rotation matrix of point O in the multi-layer refraction coordinate system relative to the camera coordinate system, P 0 is the three-dimensional point coordinate of point O in the multi-layer refraction coordinate system, r t 0 is the translation matrix of point O in the multi-layer refraction coordinate system relative to the camera coordinate system;

[0141] The position plane [u, v] of the four-dimensional parameter ray is defined on the xy plane of the multi-layer plane refraction coordinate system and coincides with its coordinate origin. The direction plane [s, t] is at a distance of one unit length from the position plane and is parallel to the position plane; among them, d is the distance from the camera optical center to the interface, d i is the distance between each interface, u i is the refractive index of each layer of medium.

[0142] Preferably, in step 4, the underwater camera imaging process based on the multi-layer plane refraction model is as follows: The light ray corresponding to the object point P passes through the refraction of m layers of media, propagates backward to the camera optical center, and intersects the imaging plane at point P w (i.e., the underwater image point). In an actual underwater imaging system, it only contains three layers of media: water, glass, and air. Since the thickness d of the glass 1 is much smaller than the propagation distance d of the light ray in water 2 , the light ray will only have a slight radial offset and the direction remains unchanged during this propagation process. Therefore, in actual calculations, the glass layer is generally ignored. According to the principle of reversibility of light paths, the underwater camera imaging process is analyzed using the reverse analysis method:

[0143] (1) According to the principle of pinhole imaging, the relationship between the object point P c (x, y, z) and the image point m = [u o , v o in the camera coordinate system can be established as: T In the formula, K is the internal parameter matrix of the camera, and any pixel

[0144]

[0145] determines a light ray passing through the optical center and the pixel; determines a light ray passing through the optical center and the pixel;

[0146] determines a light ray passing through the optical center and the pixel;

[0147] (2) In the camera coordinate system, the light ray direction I c can be expressed as:

[0148]

[0149] (3) According to the conversion relationship between coordinate systems in formula (4), the light ray direction I r can be expressed as:

[0150]

[0151] (4) The four-dimensional parameter light ray can be described as:

[0152]

[0153] (5) After the light ray propagates a certain distance d on the xy plane of the multi-layer refraction coordinate system, the expression is as follows:

[0154]

[0155] In the formula, T(d) is the mathematical expression after the light ray propagates the distance d, represents the Kronecker product;

[0156] (6) When light undergoes refraction, according to the law of refraction, the refraction process of light can be expressed as:

[0157]

[0158] where μ and μ′ are the refractive indices of the incident medium and the emerging medium respectively; R[s, t, μ, μ′] is the refraction expression; α[s, t, μ, μ′] and β[s, t, μ, μ′] are undetermined parameters for the change in the direction of the light ray after refraction. The specific solution process is as follows: Assume that the expression of the incident light ray is L(u, v, s, t) T , and the expression of the refracted light ray is L(u′, v′, s′, t′) T , then the angle of incidence of the light ray can be expressed as:

[0159]

[0160] Similarly, the angle of refraction of the light ray can be expressed as According to the law of refraction μsin(θ) = μ′sin(θ′), By combining these equations, we get:

[0161]

[0162] Since the refracted light ray and the incident light ray are in the same plane, the constraint s′t = st′ is satisfied. Therefore,

[0163] In summary, we have:

[0164]

[0165] (7) The light ray L r propagates a distance d 0 , and enters the medium μ 0 from the medium μ 1 and then, the light ray can be expressed as:

[0166]

[0167] For the light ray i L r propagates a distance d i , and then enters the medium μ i from the medium μ i+1 , the refracted light ray can be expressed as:

[0168]

[0169] (8) When the light ray passes through multiple layers of media, the refracted light ray can be expressed as:

[0170]

[0171] Preferably, in step 5, the underwater measurement model is as follows Figure 5 shown, and the specific method is as follows:

[0172] Assume that the focal lengths of the left and right cameras of the model are f, and the baseline distance is d 0 , the optical center connection line between the two cameras is parallel to the surface of the optical glass window and also parallel to the set coordinate axis; the distance between the optical center and the waterproof glass is g, and the glass thickness is h. Among them, the refractive indices of air, glass, and seawater are n 0 , n 1 , and n 2 respectively. The three-dimensional coordinates of the underwater object point in the left camera coordinate system are P, and its pixel coordinates in the left and right cameras are q l and q r ;

[0173] Assume that the four-dimensional parameter ray propagates a distance of d on the plane perpendicular to the multi-layer refraction coordinate system. At this time, the expression of the ray is:

[0174]

[0175] When the ray refracts underwater, assume that the refractive index of the incident light medium is n 1 , and the refractive index of the refracted light medium is n 2 , the incident angle is θ 1 , and the refraction angle is θ 2 . Then, according to the refraction law:

[0176]

[0177]

[0178]

[0179] Since the refracted ray and the normal of the incident ray are in the same plane, the following constraints are satisfied:

[0180] s′t = s′t (21)

[0181]

[0182] Subsequently, the direction of the ray can be obtained as:

[0183]

[0184] In summary, the propagation process of the refraction of the four-dimensional parameter ray can be represented by a matrix as:

[0185]

[0186] The line connecting an image point of the camera and the optical center determines a ray of light. The left camera point and q l = [x 1 , y 1 , the right camera point q r = [x 2 , y 2 , and the principal point position is q 0 = [x 0 , y 0 ; where the coordinate unit is mm. If it is in pixels, it can be first converted as [x 1 , y 1 = pix_width[x pix , y pix . The rays of light determined by the left and right camera image points can be respectively described as shown in the following formulas:

[0187]

[0188] When the system parameters are known, the object point coordinates can be obtained by finding the intersection of the two rays of light. Further, for the convenience of research, the principle of reversibility of the optical path is continued to analyze the underwater imaging process. Taking the left camera as an example, the process of the ray of light propagating from a certain pixel point on the image plane to the object point in water is as follows:

[0189] (1) The ray of light travels a distance g in the air and reaches the inner surface of the glass:

[0190]

[0191] (2) The ray of light enters the glass from the air and refracts:

[0192]

[0193] (3) The ray of light travels a distance h in the glass and reaches the outer surface of the glass:

[0194]

[0195] (4) The ray of light refracts at the outer surface of the glass, from the glass medium to the sea water medium:

[0196]

[0197] (5) It travels an unknown distance depth in the sea water and reaches the object point:

[0198]

[0199] Based on the above analysis, the expression of the ray of light propagation process from the pixel ray to the object point position is as follows:

[0200]

[0201] Similarly, for the right camera, we have:

[0202]

[0203] Since equations (31) and (32) represent the same object point, by solving the equations simultaneously, we can obtain the distance of the object point (the two lines intersect at depth depth), and it satisfies the following constraint equations:

[0204]

[0205] where u 1 , v 1 , s 1 , t 1 , u 2 , v 2 , s 2 and t 2 are the outgoing rays from the glass to the sea surface.

[0206] Preferably, based on the multi-layer refraction imaging model as Figure 6 shown, which consists of left and right encapsulated cameras, in step 6, the specific method is as follows:

[0207] For any point P in space, the corresponding pixels on the left and right camera planes are l m and r m . Calculate the four-dimensional rays m and m corresponding to the pixels l and on the left and right cameras after refraction through multiple interfaces and convert them into the corresponding points and vectors l a , l q , r a and r q ; Then, transform the points and vectors r a , r q in the right camera coordinate system to the left camera coordinate system, r a′ = l R r ×r a , r q′ = l R r ×r q + l t r The point P in the left camera coordinate system lies on both rays and ; Therefore, the following two constraints are satisfied:

[0208]

[0209] In Equation (34), the symbol × represents the vector cross product. By using an anti-symmetric matrix to replace the cross product, the following equation can be obtained:

[0210]

[0211] Performing singular value decomposition on Equation (35) can solve for the three-dimensional coordinates of object point P in the left camera coordinate system, that is, obtaining the three-dimensional information of the measured object point.

[0212] Embodiment

[0213] The camera is calibrated using the Zhang Zhengyou calibration method. 30 groups of chessboard photos taken are manually added using the calibration tool library, and after calculation, the internal and external parameters of the camera, rotation and translation vectors, and distortion parameters are obtained. Table 1 shows the calibration results.

[0214] Table 1 Internal and external parameters of the camera

[0215]

[0216] In order to verify the ability of three-dimensional reconstruction technology and the identification of underwater pile foundation concrete structure diseases, the environment and parameter design in other experimental processes are kept consistent, and the obtained experimental results are shown in Table 2 below.

[0217] Table 2 Experimental results

[0218]

[0219] The size of the underwater pile foundation diseases obtained through three-dimensional reconstruction and deep learning is compared with the underwater pile foundation diseases measured manually by divers, and the relative error is as Figure 7 shown.

[0220] From Figure 7 it can be seen that the error between the size data of the underwater pile foundation diseases obtained by three-dimensional reconstruction and the manually measured data is between 0% and 12.50%.

[0221] In summary, while satisfying the acquisition of clear images, the present invention uses two cameras at different positions to simultaneously capture the same target feature points in the scene. By calculating the position deviation between the corresponding image points of the two images and using the trigonometric formula, the three-dimensional information of the measured target object is obtained.

[0222] The explanations and practices disclosed in the present invention are easy to think about and understand for ordinary technical personnel in the technical field. And without departing from the principle of the present invention, several improvements and refinements can be made. Therefore, the modifications or improvements made without departing from the spirit of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A three-dimensional reconstruction method for underwater pile foundation damage, characterized in that: The steps include: Step 1: Based on the parallax theory, two cameras at different positions are used to simultaneously shoot the same target feature point in the scene, and the three-dimensional information of the target object is obtained by calculating the position deviation between the corresponding image points of the two images and using the triangulation formula; Step 2: According to the different camera positions, the measurement model is divided into two imaging measurement models: ideal and non-ideal. In the actual calculation, the posture relationship between the two cameras is calibrated, and the image planes of the two cameras are converted into the position state under ideal conditions to realize the calculation of the three-dimensional coordinates of the feature points. Step 3: Using the four-dimensional parameterization of light, it is convenient to record and track the direction and position information of light during underwater propagation to achieve modeling and analysis of the underwater camera imaging process; Step 4: Combined with the four-dimensional parameterization method of light, a multi-layer plane refraction imaging model of the underwater camera is established; Step 5: Establish an underwater measurement model; Step 6: Based on the multi-layer refraction imaging model, obtain the three-dimensional information of the object point.

2. The method for three-dimensional reconstruction of underwater pile foundation damage according to claim 1, characterized in that: In step 2, the specific method is as follows: Assume that O1 and O2 are the optical centers of the left and right cameras respectively, and the baseline distance between them is T. The projection points of any object point P0 in the space on the left and right image planes are P l and P r , where x l and x r is the horizontal coordinate of the image point on the image plane, then the parallax between the projection points is d=x l -x r ; The following formula can be obtained from the geometric relationship of space triangles: x l is the horizontal coordinate of the left image point on the plane, x r is the horizontal coordinate of the right image point on the plane, T is the baseline distance between the two, f is the focal length, and Z is the vertical distance from the object point to the camera plane; By rearranging the above formula, we can find the distance from the object point to the baseline support: x l is the horizontal coordinate of the left image point on the plane, x r is the horizontal coordinate of the right image point on the plane, T is the baseline distance between the two, f is the focal length, and Z is the vertical distance from the object point to the camera plane; From formula (2), we know that when the disparity value of the image and the posture relationship between the cameras are obtained, the coordinates of the spatial object point P in the Z direction can be obtained; similarly, the coordinates in the X and Y directions can be obtained according to the triangular similarity relationship, thereby obtaining the three-dimensional information of the object point.

3. The method for three-dimensional reconstruction of underwater pile foundation damage according to claim 1, characterized in that: In step 3, the specific method is as follows: Light d It passes perpendicularly through two parallel planes with a distance of 1 unit and intersects the two planes at points O and O respectively. d ; where the matrix [u, v] T Used to represent the position point O of the light to record the position information of the light, matrix [s, t] T Used to represent the light direction vector To record the direction information of the light; in addition, for any light in the scene, the direction vector a and the position q of the point are used to represent it. The relationship between the two representation methods is as follows: L is the unit length representation method, a is the direction vector, and q is the object point position.

4. The method for three-dimensional reconstruction of underwater pile foundation damage according to claim 1, characterized in that: In step 4, the specific method is as follows: The camera coordinate system is located at the optical center of the camera, with its z-axis parallel to the camera optical axis. n is the multi-layer plane normal and is perpendicular to the interface. The multi-layer interface normal n is the z-axis, and the cross product of the normal n and the camera z-axis is the x-axis. A multi-layer plane refraction coordinate system is constructed. The conversion relationship between the two coordinate systems is as follows: Among them, P c is the three-dimensional point coordinate in the camera coordinate system, P r is the three-dimensional point coordinate in the multi-layer refraction coordinate system, n c is the interface normal vector in the camera coordinate system, c R r is the rotation matrix of the multi-layer refraction coordinate system relative to the camera coordinate system and c t r is the translation matrix of the multi-layer refraction coordinate system relative to the camera coordinate system; the transformation relationship between the object coordinate system and the multi-layer refraction coordinate system is: P r = r R o P o + r t o (5) P r is the three-dimensional point coordinate in the multi-layer refraction coordinate system, r R0 is the rotation matrix of point O in the multi-layer refraction coordinate system relative to the camera coordinate system, and P0 is the three-dimensional point coordinate of point O in the multi-layer refraction coordinate system. r t0 is the translation matrix of point O in the multi-layer refraction coordinate system relative to the camera coordinate system; The position plane [u, v] of the four-dimensional parameter ray is defined on the xy plane of the multi-layer plane refraction coordinate system and coincides with its coordinate origin. The direction plane [s, t] is one unit length away from the position plane and is parallel to the position plane. Among them, d is the distance from the camera optical center to the interface, d i is the distance between interfaces, u i is the refractive index of each layer of medium.

5. The method for three-dimensional reconstruction of underwater pile foundation damage according to claim 4, characterized in that: In step 4, the underwater camera imaging process based on the multi-layer plane refraction model is as follows: the light corresponding to the object point P is refracted through the m layers of medium, propagates back to the optical center of the camera, and intersects the imaging plane at point P w , according to the principle of reversible optical path, the inverse analysis method is used to analyze the imaging process of underwater camera: (1) According to the pinhole imaging principle, the object point P in the camera coordinate system c (x, y, z) and the image point m = [u o , v o ] T The relationship can be established as: Where K is the intrinsic parameter matrix of the camera, and any pixel Determine a ray passing through the optical center and the pixel; (2) In the camera coordinate system, the light direction I c It can be expressed as: (3) According to the conversion relationship between coordinate systems in formula (4), the light direction I r It can be expressed as: (4) The four-dimensional parameter ray can be described as: (5) After the light propagates a certain distance d on the xy plane of the multi-layer refraction coordinate system, the expression is as follows: Where T(d) is the mathematical expression of the light after it propagates a distance d. represents the Kronecker product; (6) When light is refracted, according to the law of refraction, the refraction process of light can be expressed as: Among them, μ and μ′ are the refractive indices of the incident medium and the exit medium respectively; R[s, t, μ, μ′] is the refraction expression; α[s, t, μ, μ′] and β[s, t, μ, μ′] are the unknown parameters of the change in the direction of the light after refraction. The specific solution process is as follows: Assume that the expression of the incident light is L(u, v, s, t) T , the expression of the refracted light is L(u′, v′, s′, t′) T , then the incident angle of the light can be expressed as: Similarly, the refraction angle of light can be expressed as According to the law of refraction, μsin(θ)=μ′sin(θ′), Combined to get: Since the refracted light and the incident light remain in the same plane, the constraint s′t=st′ is satisfied. Therefore, In summary, we can get: (7) Light L r After propagating a distance d0 and entering medium μ1 from medium μ0, the light can be expressed as: For light i L r Propagation distance d i , then from the medium μ i Entering medium μ i+1 , the refracted light can be expressed as: (8) When light passes through a multilayer medium, the refracted light can be expressed as:

6. The method for three-dimensional reconstruction of underwater pile foundation damage according to claim 1, characterized in that: In step 5, the specific method is as follows: Assume that the focal length of the left and right cameras of the model is f, the baseline distance is d0, the optical center line between the two cameras is parallel to the optical glass window surface and also parallel to the set coordinate system axis; the distance between the optical center and the waterproof glass is g, and the thickness of the glass is h. The refractive indices of air, glass and seawater are n0, n1 and n2 respectively. The three-dimensional coordinates of the underwater object point in the left camera coordinate system are P, and its pixel coordinates in the left and right cameras are q respectively. l and q r ; Assuming that the propagation distance of the four-dimensional parameter light on the plane of the vertical multi-layer refraction coordinate system is d, the expression of the light is: When light is refracted underwater, assuming that the refractive index of the incident light medium is n1, the refractive index of the refracted light medium is n2, the incident angle is θ1, and the refraction angle is θ2, then the law of refraction gives: Since the refracted ray and the incident ray normal are in the same plane, the following constraints are satisfied: s′t=st′ (21) Then the direction of the light can be obtained as: In summary, the propagation process of refraction of four-dimensional parameter light can be expressed by matrix: The line connecting an image point of the camera and the optical center determines a ray, the left camera point and q l = [x1, y1], right camera point q r = [x2, y2], the principal point position is q0 = [x0, y0]; where the coordinate unit is mm, if it is pixel, it can be converted first [x1, y1] = pix_width[x pix ,y pix ], the light rays determined by the left and right camera image points can be described as follows: When the system parameters are known, the coordinates of the object point can be obtained by the intersection of two rays. The reversible principle of the optical path is used to analyze the underwater imaging process. Taking the left camera as an example, the process of light being transmitted from a pixel point on the image plane to the object point in the water is as follows: (1) The distance g that the light travels in the air reaches the inner surface of the glass: (2) When light enters glass from air, it refracts: (3) The light travels a distance h in the glass and reaches the outer surface of the glass: (4) The light is refracted at the glass surface, from the glass medium to the seawater medium: (5) Propagate an unknown distance depth in seawater and reach the object point: Based on the above analysis, the light propagation process from pixel light to object point position is expressed as follows: Similarly, for the right camera: Since equations (31) and (32) represent the same object point, the distance between the object points can be obtained by solving them together, and it satisfies the following constraint equation: Among them, u1, v1, s1, t1, u2, v2, s2 and t2 are the outgoing light rays from the glass to the sea surface.

7. The method for three-dimensional reconstruction of underwater pile foundation damage according to claim 1, characterized in that: In step 6, the specific method is as follows: For any point P in space, the corresponding pixel on the left and right camera phase planes is l m and r m , respectively calculate the left and right camera pixels l m and r m The four-dimensional light that reaches the last layer of medium after being refracted through the multi-layer interface and and convert it into the corresponding point and vector l a , l q 、r a and r q ; Then, the point and vector r in the right camera coordinate system a 、r q Convert to the left camera coordinate system, r a′= l R r × a , r q′= l R r × r q+ l t r Point P in the left camera coordinate system is also located on the light and Therefore, the following two constraints are satisfied: In formula (34), the symbol × represents the vector cross product. Using the antisymmetric matrix instead of the cross product, we can get the following formula: By performing singular value decomposition on formula (35), the three-dimensional coordinates of the object point P in the left camera coordinate system can be obtained, that is, the three-dimensional information of the measured object point can be obtained.

Images