A 3D point cloud measurement method for muddy water based on infrared diffraction spots and binocular vision

Abstract

The present invention provides a muddy water three-dimensional point cloud measurement method based on infrared diffraction spots and binocular vision, the method comprising the following steps: (1) constructing an underwater optical three-dimensional shape measurement system based on infrared diffraction spots and binocular vision; (2) constructing an underwater infrared camera imaging model, calibrating the refraction parameters of the underwater infrared camera system, and correcting the deviation caused by different refractive indices of the medium; (3) projecting an infrared diffraction spot on the surface of the object to be measured as a carrier of the shape information of the surface of the object to be measured; (4) the camera collects images, and performs image enhancement processing on the collected images based on the muddy water image degradation model; (5) obtaining local optical three-dimensional shape point cloud data of the object surface at a certain stage by calculation, and moving the optical measurement device to obtain local optical three-dimensional shape point cloud data of the object surface at each stage. The present invention overcomes the problem of light refraction in underwater measurement, eliminates the error caused by refraction, and improves the accuracy of the underwater optical measurement method.

Description

Technical Field

[0001] The invention relates to the field of underwater high-precision measurement, and in particular to a muddy water three-dimensional point cloud measurement method based on infrared diffraction spots and binocular vision. Background Art

[0002] Optical-based 3D shape measurement methods include 3D digital speckle correlation method, grating projection method, line structured light method, etc., which use the principle of triangulation to measure 3D shape. The advantages of optical measurement methods are non-contact, high measurement accuracy and high speed.

[0003] The main problems faced by underwater measurement technology based on optical measurement are: 1. The natural texture characteristics of the surface of the underwater object to be measured are not enough; 2. The error caused by the refraction of multiple media underwater; 3. The imaging quality is poor in turbid water conditions.

[0004] Through infrared diffraction technology, optical marks can be placed on the surface of underwater objects to solve the problem of weak natural texture of underwater objects. By establishing an underwater refraction imaging model, the imaging error caused by light refraction can be eliminated. Through image enhancement technology, the impact of muddy water imaging on image quality can be eliminated. Furthermore, optical measurement methods can be applied to the field of high-precision underwater measurement and are suitable for more underwater measurement scenarios. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a turbid water three-dimensional point cloud measurement method based on infrared diffraction spots and binocular vision, which can provide surface information to be measured through infrared diffraction spots, construct an underwater refraction correction model and a turbid water imaging model to correct the image, and use a three-dimensional measurement method based on binocular vision to obtain high-precision three-dimensional point cloud information of the underwater object shape, overcome the problem of light refraction in underwater measurement, eliminate the error caused by refraction, and improve the accuracy of the underwater optical measurement method.

[0006] In order to solve the problems of the prior art, the technical solution adopted by the present invention is:

[0007] A method for measuring muddy water three-dimensional point cloud based on infrared diffraction spots and binocular vision comprises the following steps:

[0008] Step 1: Build an underwater optical 3D shape measurement system, place and fix the infrared camera, camera waterproof cover, camera holder, and DOE laser according to the measurement plan, and calibrate the intrinsic and extrinsic parameters of the infrared camera using a calibration plate;

[0009] Step 2, calibrating the refraction parameters of the underwater camera according to the camera intrinsic parameters and extrinsic parameters calibrated in step 1, and obtaining an underwater refraction imaging model, wherein the underwater refraction imaging model is used to correct the error caused by different refractive indices when light passes through different media;

[0010] Step 3, setting the parameters of the DOE laser so that the diffraction spot projected by the DOE laser on the surface of the underwater object can cover the field of view of the infrared camera, and the size of a single spot occupies 5-10 pixels in the infrared camera screen, and the diffraction spot serves as the morphological information carrier of the surface of the object to be measured;

[0011] Step 4, collecting infrared spot images through an infrared camera, and performing image enhancement processing on the collected images based on the underwater image degradation model to obtain high-quality infrared diffraction spots;

[0012] Step 5, obtaining the local optical three-dimensional shape point cloud data of the object surface at a certain stage by calculation, and moving the optical measuring device to obtain the local optical three-dimensional shape point cloud data of the object surface at each stage.

[0013] As an improvement, the internal calibration parameters include: lens focal length f, principal point position (u0, v0), pixel size (S x , S y ) and lens distortion parameters, the lens distortion parameters include 6-order radial distortion parameters K1, K2, K3, K4, K5, K6 and 2-order tangential distortion parameters P1, P2; the external parameter calibration parameters include: the rotation matrix R and translation matrix t between the two infrared cameras, the rotation matrix R includes r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 Six parameters, the translation matrix t includes three parameters t1, t2, and t3.

[0014] Furthermore, in step 2, the refraction parameters of the underwater camera are calibrated, and the specific steps are as follows: when the number of the units of the infrared camera in step 1 is 2, the two infrared cameras (C0, C1) are placed in a waterproof cover with a transparent flat observation window, and the light from point P is imaged on the photosensitive target surface through the three media of water, glass observation window and air. The refractive indexes of air, glass and water are n1, n2 and n3 respectively. If no refraction correction is performed, the obtained three-dimensional point is P′, and ||PP′|| is the error;

[0015] In the left camera coordinate system C0-XYZ, C0 = (0, 0, 0), C1 = t = (t x , t y , t z ), where t is the translation matrix in the camera extrinsic parameters. Using the coordinates of point P' in the left camera (C0) and the right camera (C1), we can get the light vectors l1 and l1':

[0016]

[0017] Then in the left camera coordinate system C0-XYZ, the equation of the line where the l1 and l1′ vectors are located is:

[0018]

[0019] According to the thickness of the air layer D1 and the normal vector n of the refraction surface, the equation of the refraction surface Π1 can be obtained:

[0020] Π1:n T X+D1=0

[0021] The intersection points of light ray l1 and light ray l1′ with refractive surface Π1 are P1 and P1′ respectively:

[0022]

[0023] When light passes through the refractive surface Π1 of air and glass, it will be refracted due to the different refractive indices. According to Snell's law, the refracted light vectors l2 and l2' are respectively:

[0024]

[0025] Assuming that the coordinates of P1 and P1′ are (X1, Y1, Z1) and (X1′, Y1′, Z1′), the equations of the lines on which the vectors l2 and l2′ lie are:

[0026]

[0027] According to the glass layer thickness D2 and the refractive surface normal vector n, the equation of the refractive surface Π2 can be obtained:

[0028] Π2:n T X+D1+D2=0

[0029] The intersection points of light ray l2 and light ray l2′ with the refractive surface Π2 are P2 and P2′ respectively:

[0030]

[0031] Similarly, the refracted light vectors l3 and l3′ are:

[0032]

[0033] By using the least squares method to calculate the intersection of l3 and l3′, the real object point P can be obtained, that is, the influence of refraction is eliminated. Therefore, the calibrated refraction model should include the following parameters: the distance D1 from the optical center of the camera to the inner surface of the observation window, the thickness of the observation window D2, the normal vector n of the observation window plane = (n x , ny , n z ).

[0034] Furthermore, in order to calibrate the refraction parameters in the refraction model, it is necessary to calibrate the refraction model, that is, to determine the equations of the refraction surfaces inside and outside the observation window in the camera coordinate system. The methods used include but are not limited to: arranging ordinary speckles, marker points, and fluorescent speckles on the inner and outer surfaces of the observation window.

[0035] Furthermore, the optical three-dimensional shape measurement method used in the calculation in step 5 is a three-dimensional digital speckle correlation method, a grating line projection method (surface structured light method) or a line structured light method.

[0036] Beneficial effects:

[0037] Compared with the prior art, the present invention is a muddy water three-dimensional point cloud measurement method based on infrared diffraction spot and binocular vision. Through infrared diffraction technology, optical marks can be placed on the surface of underwater objects to be measured, solving the problem of weak natural texture of underwater objects. By establishing an underwater refraction imaging model, the imaging error caused by light refraction is eliminated. Through image enhancement technology, the influence of muddy water imaging on image quality is eliminated. Furthermore, the optical measurement method can be applied to the field of high-precision underwater measurement and is suitable for more underwater measurement scenarios.

[0038] The specific advantages are as follows:

[0039] 1. In order to overcome the problem of weak natural texture and insufficient feature information on the surface of underwater objects, the infrared diffraction spot technology is innovatively used. Compared with visible light, infrared light has strong penetrating ability and is more adaptable to muddy water environments. In addition, the spot generated by this technology is a planar array, which has the advantages of large 3D reconstruction area and fast speed.

[0040] 2. The innovative use of underwater refraction correction technology based on binocular vision measurement can overcome the problem of light refraction in underwater measurement, eliminate the error caused by refraction, obtain the real three-dimensional point cloud of the object, and improve the accuracy of the underwater optical measurement method. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the muddy water three-dimensional point cloud measurement method based on infrared diffraction spots and binocular vision of the present invention;

[0042] Figure 2 This is the refraction imaging model diagram of the underwater camera;

[0043] Figure 3 Schematic diagram of the placement of DOE infrared laser and infrared camera;

[0044] Figure 4 is the fluorescent speckle on the surface of the observation window;

[0045] Figure 5 is the diffraction spot;

[0046] Figure 6 Image enhancement processing: before processing (left), after processing (right);

[0047] Figure 7 These are the results of refraction correction: refraction error not eliminated (top), refraction error eliminated (bottom), and true position (bottom). DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with specific implementation cases.

[0049] Example 1

[0050] A muddy water three-dimensional point cloud measurement method based on infrared diffraction spots and binocular vision, wherein the optical measurement method used is a three-dimensional digital speckle correlation method, and the speckle used is an infrared diffraction spot.

[0051] The specific steps are as follows:

[0052] (1) Place and fix the infrared camera, camera waterproof cover, camera mount, and DOE laser according to the measurement plan. Use the calibration board to calibrate the intrinsic and extrinsic parameters of the infrared camera. The intrinsic calibration parameters include: lens focal length f, principal point position (u0, v0), pixel size (S x , S y ) and lens distortion parameters, wherein the lens distortion parameters include 6-order radial distortion parameters K1, K2, K3, K4, K5, K6 and 2-order tangential distortion parameters P1, P2. The external calibration parameters include: the rotation matrix R and the translation matrix t between the two infrared cameras, wherein R includes r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 Six parameters, t includes t1, t2, and t3;

[0053] (2) Fluorescent speckles are sprayed on the inner and outer surfaces of the observation window in sequence, and the topography of the observation window surface (refractive surface) is measured using binocular visual imaging technology in air. The refraction parameters of the underwater camera refraction imaging model can be obtained: the distance D1 from the camera optical center to the inner surface of the observation window, the thickness of the observation window D2, and the normal vector n of the observation window plane, which are used to correct the error caused by the different refractive indices when the light passes through different media, such as Figure 4 As shown;

[0054] (3) Adjust the parameters of the DOE laser so that the diffraction spot projected by the DOE laser on the surface of the underwater object can cover the entire field of view of the infrared camera, and the size of a single spot occupies 5-10 pixels in the infrared camera image, such as Figure 5 As shown;

[0055] (4) Collect infrared spot images through an infrared camera, and use a dark channel prior algorithm to perform image enhancement processing on the collected images to obtain high-quality infrared diffraction spots;

[0056] (5) Based on the digital image correlation method, stereo matching is performed on the spot images of a certain stage collected by the left and right cameras in step (4), and the camera internal and external parameters and refraction parameters obtained in steps (1) and (2) are combined to perform three-dimensional reconstruction on the calculated points matched one by one in the image to obtain the local optical three-dimensional shape point cloud data of the object surface, and then the binocular camera is moved to obtain the local optical three-dimensional shape point cloud data of other areas; if the influence of underwater refraction on three-dimensional reconstruction is not corrected, there will be a shape measurement error of up to 40mm. The muddy water three-dimensional point cloud measurement method based on infrared diffraction spot and binocular vision proposed in this patent can effectively eliminate the error caused by refraction and reduce the shape measurement error to less than 2mm. Through the displacement experiment, it is proved that this method can reduce the displacement measurement error from 22% to 3.6%.

Claims

1. A method for measuring muddy water three-dimensional point cloud based on infrared diffraction spots and binocular vision, characterized in that: The following steps are involved: Step 1: Build an underwater optical 3D shape measurement system, place and fix the infrared camera, camera waterproof cover, camera holder, and DOE laser according to the measurement plan, and calibrate the intrinsic and extrinsic parameters of the infrared camera using a calibration plate; Step 2: calibrate the refraction parameters of the underwater camera according to the camera intrinsic parameters and extrinsic parameters calibrated in step 1 to obtain an underwater refraction imaging model, wherein the underwater refraction imaging model is used to correct the error caused by different refractive indices when light passes through different media. Among them, the refraction parameters of the underwater camera are calibrated, and the specific steps are as follows: when the number of the units of the infrared camera described in step 1 is 2, the two infrared cameras (C0, C1) are placed in a waterproof cover with a transparent flat observation window, and the light from point P is imaged on the photosensitive target surface through the three media of water, glass observation window and air. The refractive indices of air, glass and water are n1, n2 and n3 respectively. If no refraction correction is performed, the obtained three-dimensional point is P′, and ||PP′|| is the error; in the left camera coordinate system C0-XYZ, C0=(0,0,0), C1=t=(t x , t y , t z ), where t is the translation matrix in the camera extrinsic parameters. Using the coordinates of point P' in the left camera (C0) and the right camera (C1), we can get the light vectors l1 and l1': Then in the left camera coordinate system C0-XYZ, the equation of the line where the l1 and l1′ vectors are located is: According to the thickness of the air layer D1 and the normal vector n of the refraction surface, the equation of the refraction surface Π1 can be obtained: Π1:n T X+D1=0 The intersection points of light ray l1 and light ray l1′ with refractive surface Π1 are P1 and P1′ respectively: When light passes through the refractive surface Π1 of air and glass, it will be refracted due to the different refractive indices. According to Snell's law, the refracted light vectors l2 and l2' are respectively: Assuming that the coordinates of P1 and P1′ are (X1, Y1, Z1) and (X1′, Y1′, Z1′), the equations of the lines on which the vectors l2 and l2′ lie are: According to the glass layer thickness D2 and the refractive surface normal vector n, the equation of the refractive surface Π2 can be obtained: Π2:n T X+D1+D2=0 The intersection points of light ray l2 and light ray l2′ with the refractive surface Π2 are P2 and P2′ respectively: Similarly, the refracted light vectors l3 and l3′ are: By using the least squares method to calculate the intersection of l3 and l3′, the real object point P can be obtained, that is, the influence of refraction is eliminated. Therefore, the calibrated refraction model should include the following parameters: the distance D1 from the optical center of the camera to the inner surface of the observation window, the thickness of the observation window D2, the normal vector n of the observation window plane = (n x , n y , n z ); Step 3, setting the parameters of the DOE laser so that the diffraction spot projected by the DOE laser on the surface of the underwater object can cover the field of view of the infrared camera, and the size of a single spot occupies 5-10 pixels in the infrared camera screen, and the diffraction spot serves as the morphological information carrier of the surface of the object to be measured; Step 4, collecting infrared spot images through an infrared camera, and performing image enhancement processing on the collected images based on the underwater image degradation model to obtain high-quality infrared diffraction spots; Step 5, obtaining the local optical three-dimensional shape point cloud data of the object surface at a certain stage by calculation, and moving the optical three-dimensional shape measurement system to obtain the local optical three-dimensional shape point cloud data of the object surface at each stage.

2. The method for measuring muddy water three-dimensional point cloud based on infrared diffraction spots and binocular vision according to claim 1 is characterized in that: The internal calibration parameters include: lens focal length f, principal point position (u0, v0), pixel size (S x , S y ) and lens distortion parameters, the lens distortion parameters include 6-order radial distortion parameters K1, K2, K3, K4, K5, K6 and 2-order tangential distortion parameters P1, P2; the external parameter calibration parameters include: the rotation matrix R and translation matrix t between the two infrared cameras, the rotation matrix R includes r 11 、r 12 、r 13 、r 21 、r 22 、r 23 、r 31 、r 32 、r 33 Six parameters, the translation matrix t includes three parameters t1, t2, and t3.

3. The method for measuring muddy water three-dimensional point cloud based on infrared diffraction spots and binocular vision according to claim 1 is characterized in that: In order to calibrate the refraction parameters in the refraction model, the refraction model needs to be calibrated, that is, the equations of the refraction surfaces inside and outside the observation window in the camera coordinate system are determined. The methods used include but are not limited to: arranging ordinary speckles, marker points, and fluorescent speckles on the inner and outer surfaces of the observation window.

4. The method for measuring muddy water three-dimensional point cloud based on infrared diffraction spots and binocular vision according to claim 1 is characterized in that: The optical three-dimensional shape measurement method used in the calculation in step 5 is a three-dimensional digital speckle correlation method, a grating line projection method or a line structured light method.

Images