Vision and line structured light fused underwater three-dimensional measurement method

Abstract

The invention discloses a vision and line structured light fused underwater three-dimensional measurement method. The method comprises the following steps: step 1, preprocessing an acquired image; 2, system parameter calibration is carried out on the preprocessed in-air color calibration image and the preprocessed in-air center line structure light calibration image, and internal and external parameters of a camera and a line structure light plane equation are obtained; 3, establishing a three-dimensional refraction model for the underwater color three-dimensional measurement object image and the underwater line structure light measurement object image, and performing affine transformation of the three-dimensional refraction model; 4, performing laser line center extraction on the transformed underwater line structured light measurement object image, and substituting the laser line center into the calibrated line structured light plane equation to perform point cloud reconstruction; and step 5, performing color processing on the underwater color three-dimensional measurement object image, and mapping the image to three-dimensional point cloud data. According to the invention, through affine transformation of the multi-medium refraction image and point cloud mapping of color multi-channel equalization, the precision and visual effect of underwater three-dimensional measurement are significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of underwater three-dimensional measurement technology, and in particular to an underwater three-dimensional measurement method that combines vision and line structured light. Background Art

[0002] Line structured light 3D measurement technology has been widely used in many industrial and scientific fields due to its high precision and high efficiency. In scenes with obvious laser features, accurately extracting the characteristic curve of laser projection can significantly improve the accuracy of 3D measurement.

[0003] However, when line structured light 3D measurement technology is applied to complex media environments such as underwater and glass, the situation becomes complicated. Due to the optical properties of these media, the propagation path of the laser will be affected in a complex way, which in turn restricts the accuracy of 3D measurement.

[0004] In terms of underwater 3D reconstruction, existing underwater 3D reconstruction models have certain limitations. Most models only contain the geometric features of objects and lack more comprehensive information. At the same time, the existing technology for underwater image color enhancement is also limited to the optimization of color information on the plane and cannot reflect the geometric structure of the target in three-dimensional space. Due to its own characteristics, the enhancement method of two-dimensional images cannot accurately combine color information with the three-dimensional surface geometric details of the object, which leads to the inability to present a good sense of reality when facing complex-shaped objects. Summary of the invention

[0005] In view of this, the present invention proposes an underwater 3D measurement method integrating vision and line structured light to solve the problems in the prior art of low accuracy of underwater measurement using line structured light and poor integration of underwater image color and geometric structure.

[0006] The specific technical solutions of the present invention are as follows:

[0007] An underwater three-dimensional measurement method integrating vision and line structured light, comprising:

[0008] Step 1, preprocessing the collected images, including classification processing of color calibration images in air, line structured light calibration images in air, underwater color 3D measurement object images, and underwater line structured light measurement object images, as well as corresponding Gaussian filtering and morphological operations;

[0009] Step 2, calibrating the system parameters of the preprocessed color calibration image in air and the line structured light calibration image in air to obtain the internal and external parameters of the camera and the line structured light plane equation;

[0010] Step 3, establishing a three-dimensional refraction model and performing affine transformation on the underwater color three-dimensional measurement object image and the underwater line structured light measurement object image to correct image distortion caused by refraction;

[0011] Step 4, extracting the center of the laser line from the transformed underwater line structured light measurement object image, and substituting it into the calibrated line structured light plane equation to reconstruct the point cloud;

[0012] Step 5: Perform color processing on the underwater color three-dimensional measurement object image, and map the processed color information into three-dimensional point cloud data.

[0013] Specifically, in step 1, the line structured light measurement image in the air and the underwater line structured light measurement object image are converted into monochrome images, binarized, and subjected to morphological operations to highlight the features of the laser line.

[0014] Specifically, in step 2, the internal and external parameters of the camera are obtained through camera calibration. The internal parameters include focal length and principal point coordinates, and the external parameters include rotation and translation parameters of the camera.

[0015] Specifically, in step 2, the least squares method is used to fit the line structured light plane equation.

[0016] Specifically, in step 3, a multi-medium refraction model is established according to Snell's law, and an affine transformation is performed to correct image distortion.

[0017] Specifically, in step 4, when extracting the center of the laser line of the transformed laser image, the center position of the laser line is determined by finding the grayscale peak of the laser line or using an edge detection algorithm; the pixel coordinates of the center of the laser line are converted into point cloud world coordinates using the camera imaging principle; the camera imaging principle includes the use of the camera intrinsic parameter matrix, rotation matrix, and translation vector; the point cloud reconstruction process uses the camera imaging model to substitute the extracted pixel coordinates into the calibrated line structured light plane equation to realize the conversion from two-dimensional image information to three-dimensional point cloud data.

[0018] Specifically, in step 5, the blue cast of the underwater image is reduced by separating and balancing the red and green channels in the RGB color space; the color mapping process extracts the processed image color information and accurately maps it to the three-dimensional point cloud data, ensuring that the color information can accurately correspond to each point in the point cloud data; the color mapping process solves the color consistency problem between the underwater image and the point cloud data by avoiding excessive enhancement of noise while enhancing the local details of the image, thereby improving the visual effect of underwater three-dimensional reconstruction.

[0019] Specifically, in step 2, the line structured light plane equation is obtained by extracting the intersection points between the center lines of the laser lines on multiple line structured light calibration images in the air and the circular intersection lines of the circular calibration plate, converting the two-dimensional points into three-dimensional points, and then fitting them using the least squares method.

[0020] Specifically, in step 1, when preprocessing the line structured light image, it is first converted into a monochrome image and binarized, with the constrained binary grayscale value being 125. Then, an erosion operation with the convolution kernel set to (2, 2) is performed to reduce the scattering noise near the laser line, and then a dilation operation is performed to maintain the integrity of the laser line.

[0021] Specifically, in step 3, the affine transformation is achieved by calculating 8 position parameters of the affine transformation matrix to restore the image taken underwater to the image coordinates in the air.

[0022] The beneficial effects of the present invention are:

[0023] (1) The present invention proposes a multi-medium refraction image affine transformation method in step 3, which can convert an image in water into an image in air. This technological breakthrough significantly reduces the impact of the parallax effect in underwater images on the accuracy of three-dimensional measurement, thereby greatly improving the accuracy of three-dimensional measurement.

[0024] (2) The present invention proposes a color multi-channel balanced point cloud mapping method in step 5. This method enhances the local details of the image while avoiding excessive enhancement of noise, ensuring the balance of image brightness and contrast without causing visual distortion. By integrating this method into the point cloud data, the visual effect of underwater 3D reconstruction is significantly improved, making the reconstruction result clearer, more realistic, and richer in details. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 It is a schematic diagram of the process of the underwater three-dimensional measurement method of the present invention that combines vision and line structured light;

[0027] Figure 2 A schematic diagram of the structure of the image captured underwater in the present invention being affected by the path of refraction;

[0028] Figure 3 It is a structural schematic diagram of the camera imaging model of the present invention. DETAILED DESCRIPTION

[0029] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0030] The present invention proposes an underwater three-dimensional measurement method combining vision and line structured light, which specifically comprises the following steps:

[0031] Step 1: Preprocess the collected images:

[0032] Build laser line equipment and cameras to collect images. Classify and process the collected images into color calibration images in the air, line structured light calibration images in the air, color 3D measurement object images underwater, and line structured light measurement object images underwater. Among them, the image in the air is used to calibrate the spatial position of the camera. The color calibration image in the air is used to calibrate the camera internal parameters; the line structured light calibration image in the air is used to calibrate the line structured light plane equation; the underwater color 3D measurement object image is used to extract the color mapping to the 3D point cloud; the underwater line structured light measurement object image is used to substitute the line structured light plane equation to obtain the 3D point cloud. In other words, it is used to accurately convert the laser line pixel coordinates of the underwater line structured light image into the 3D point cloud coordinates.

[0033] For color images (color calibration images in the air, color 3D measurement object images underwater), Gaussian filtering is used to reduce image noise. This is because color images are easily affected by various interferences and generate noise. Gaussian filtering can effectively smooth the image and reduce the impact of noise by weighted averaging of image pixels.

[0034] In terms of processing line structured light images (line structured light positioning images in the air, underwater line structured light measurement object images), the format of the laser image is also an RGB three-channel color image. First, it is converted to a monochrome image (grayscale single-channel format) and the image is binarized, where the constrained binarization grayscale value is 125. Line structured light images remove the laser line on the image, and there will be other dark light source noise. Binarization simplifies the image into an image with only two grayscale values, eliminating all noise below the grayscale value in the image, which helps to highlight the characteristics of the laser line and facilitates the subsequent extraction of the laser line.

[0035] Then, morphological operations are performed on the binary monochrome image. The scattering noise near the laser line is first reduced by setting the convolution kernel to (2, 2) for the erosion operation. The erosion operation can shrink the boundary of the target object (here, the laser line) in the image, thereby removing some small noise points around the laser line. After that, the dilation operation is performed on the set convolution kernel (2, 2) to maintain the integrity of the laser line. The dilation operation is the opposite of the erosion operation. It can expand the boundary of the target object and restore the complete shape of the laser line after removing the noise, ensuring the accuracy of the laser line in subsequent processing.

[0036] Step 2: calibrate the system parameters of the calibration images processed in step 1 (color calibration images in air, linear structure calibration images in air):

[0037] The camera is calibrated on the color calibration image in air in step 1 to obtain the internal and external parameters of the camera. The internal parameters of the camera include focal length, principal point coordinates, etc., and the external parameters include the rotation and translation parameters of the camera. These parameters are crucial for calculating the conversion of image two-dimensional coordinates to three-dimensional coordinates. Through accurate camera calibration, an accurate relationship between image pixel coordinates and actual world coordinates can be established.

[0038] Next, the intersection points between the center line of the laser line and the circular intersection line of the circular calibration plate on multiple line structured light calibration images in the air are extracted. The two-dimensional points are then converted to three-dimensional points, and the least squares method is used to fit the line structured light plane equation. The least squares method finds the best function match for the data by minimizing the sum of squares of the errors. Here, it can fit an accurate line structured light plane equation based on the three-dimensional coordinates of multiple intersection points, which is used for the subsequent line structured light measurement of the object image laser line to world coordinate conversion, thereby ensuring the accurate representation of the laser line in three-dimensional space.

[0039] Step 3: Establish a three-dimensional refraction model and perform affine transformation on the measurement object image of step 1 (underwater color three-dimensional measurement object image, underwater line structured light measurement object image):

[0040] The present invention establishes a method combining a three-dimensional measurement model that integrates vision and laser lines with a multi-medium refraction model. The underwater environment is multi-medium, and light will refract when propagating in different media, which will cause image distortion. According to the law of refraction in multi-medium according to Snell's law, a multi-medium refraction model is established, and the image is subjected to affine transformation. Snell's law describes the refraction relationship of light at the interface of different media. This law can be used to accurately calculate the changes in the propagation path of light in different underwater media, thereby performing a multi-medium refraction affine transformation on the underwater distorted image to restore it to an undistorted image in the air. This affine transformation can correct the deviation of image shape and position caused by refraction, laying the foundation for subsequent accurate three-dimensional measurement.

[0041] The law of refraction that explains the change in the path of light is as follows:

[0042]

[0043] Among them, n a Represents the refractive index of the air medium; n h Represents the refractive index of organic glass medium; n w represents the refractive index of water; θ a represents the incident angle of the line structured light in the air; θ h represents the incident angle of the line structured light in the organic glass; θ w Represents the incident angle of the line structured light in water.

[0044] The formula for restoring an underwater image to air is:

[0045]

[0046] Among them, (u, v, 1) is the image coordinate on land, (u′, v′, 1) is the image coordinate underwater, M is the affine transformation matrix, m1, m2, m3, m4, m5, m6, m7, m8 are the 8 position parameters of the affine transformation matrix.

[0047] m1, m2, m3, m4, m5, m6, m7, m8 can be obtained by extracting the image coordinate points. The calculation method is as follows:

[0048]

[0049] Images taken underwater are affected by the path of refraction. Figure 2 As shown, where η a , η g , η w are the refractive indices of air, glass and water respectively; θ a ,θ g ,θ w are the refraction angles of air, glass, and water respectively.

[0050] Step 4: extract the laser line center of the laser image transformed in step 3 and substitute it into the equation calibrated in step 2 to reconstruct the point cloud:

[0051] Using the camera imaging principle, first extract the pixel coordinates of the center of the laser line of the transformed image in step 3. This requires precise analysis of the laser line image, such as by finding the grayscale peak of the laser line or using an edge detection algorithm to determine the center position of the laser line. Then substitute the extracted pixel coordinates into the line structured light plane equation calibrated in step 2 according to the camera imaging model. The camera imaging model describes the mapping relationship between the coordinates of an object in the three-dimensional world and the pixel coordinates on the image plane. Through this model and the calibrated line structured light plane equation, the pixel coordinates can be converted to point cloud world coordinates. Among them, for the pixel coordinates u and v on the image, they are converted to world coordinates through the camera intrinsic and extrinsic parameters. This process realizes the conversion from two-dimensional image information to three-dimensional point cloud data, which is a key step in building a three-dimensional model of underwater objects.

[0052] Camera imaging model such as Figure 3 As shown, where P represents the imaging target point, X c , Y c , Z c is the camera coordinate system, O is the image center, O c is the origin of the camera coordinate system.

[0053] World coordinates (x w ,y w , z w ) and pixel coordinates are as follows:

[0054]

[0055] Among them, z c Indicates the depth value of the imaging target point in the camera coordinate system; f x Indicates the horizontal focal length of the camera; f y Indicates the vertical focal length of the camera; in general, f x =f y , (u, v) represents the image center coordinates; R represents the rotation matrix; t represents the translation vector.

[0056] is the camera intrinsic parameter matrix.

[0057] Represents the transformation from the camera coordinate system to the world coordinate system. C x and C y are the image center coordinates of the camera.

[0058] Step 5, point cloud color mapping;

[0059] For the underwater color three-dimensional measurement object image in step 3, the blue bias phenomenon is first reduced by separating and balancing the red and green channels in the RGB color space. In underwater environments, due to reasons such as light absorption and scattering, images often appear blue. By processing the red and green channels, the local details of the image can be enhanced while avoiding excessive noise enhancement without causing excessive brightness or contrast distortion. The color information of the processed image is then extracted and accurately mapped to the three-dimensional point cloud data. This process ensures that the color information can accurately correspond to each point in the point cloud data, thereby solving the color consistency problem between the underwater image and the point cloud data, and significantly improving the visual effect of underwater three-dimensional reconstruction. Through color mapping, the reconstructed three-dimensional model is closer to the color of the real object in appearance, which improves the visualization quality of the three-dimensional measurement results.

[0060] The beneficial effects of the present invention are:

[0061] (1) The multi-medium refraction image affine transformation method in step 3 can convert the underwater image into the air image, greatly reducing the three-dimensional accuracy problem caused by the parallax of the underwater image, thereby improving the measurement accuracy.

[0062] (2) The color multi-channel balanced point cloud mapping method in step 5 can enhance the local details of the image while avoiding excessive noise enhancement, and will not cause excessive brightness or contrast distortion. After being fused with the point cloud data, it can significantly improve the visual effect of underwater 3D reconstruction.

[0063] (3) The present invention increases the color information of the underwater three-dimensional model and improves the recognizability of the point cloud model.

[0064] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An underwater three-dimensional measurement method combining vision and line structured light, characterized in that: include: Step 1, preprocessing the collected images, including classification processing of color calibration images in air, line structured light calibration images in air, underwater color 3D measurement object images, and underwater line structured light measurement object images, as well as corresponding Gaussian filtering and morphological operations; Step 2, calibrating the system parameters of the preprocessed color calibration image in air and the line structured light calibration image in air to obtain the internal and external parameters of the camera and the line structured light plane equation; Step 3, establishing a three-dimensional refraction model and performing affine transformation on the underwater color three-dimensional measurement object image and the underwater line structured light measurement object image to correct image distortion caused by refraction; Step 4, extracting the center of the laser line from the transformed underwater line structured light measurement object image, and substituting it into the calibrated line structured light plane equation to reconstruct the point cloud; Step 5: Perform color processing on the underwater color three-dimensional measurement object image, and map the processed color information into three-dimensional point cloud data.

2. The underwater three-dimensional measurement method combining vision and line structured light according to claim 1, characterized in that: In the step 1, the line structure light measurement image in the air and the underwater line structure light measurement object image are converted into monochrome images, binarized and subjected to morphological operations to highlight the features of the laser line.

3. The underwater three-dimensional measurement method combining vision and line structured light according to claim 1, characterized in that: In step 2, the internal and external parameters of the camera are obtained through camera calibration, where the internal parameters include focal length and principal point coordinates, and the external parameters include rotation and translation parameters of the camera.

4. The underwater three-dimensional measurement method combining vision and line structured light according to claim 1, characterized in that: In the step 2, the line structured light plane equation is obtained by fitting using the least square method.

5. The underwater three-dimensional measurement method combining vision and line structured light according to claim 1, characterized in that: In step 3, a multi-medium refraction model is established according to Snell's law, and an affine transformation is performed to correct image distortion.

6. The underwater three-dimensional measurement method combining vision and line structured light according to claim 1, characterized in that: In the step 4, when extracting the center of the laser line from the transformed laser image, the center position of the laser line is determined by finding the grayscale peak of the laser line or using an edge detection algorithm; the pixel coordinates of the center of the laser line are converted into point cloud world coordinates using the camera imaging principle; the camera imaging principle includes the use of the camera intrinsic parameter matrix, rotation matrix, and translation vector; the point cloud reconstruction process uses the camera imaging model to substitute the extracted pixel coordinates into the calibrated line structured light plane equation to achieve the conversion from two-dimensional image information to three-dimensional point cloud data.

7. The underwater three-dimensional measurement method combining vision and line structured light according to claim 1, characterized in that: In step 5, the blue cast of the underwater image is reduced by separating and balancing the red and green channels in the RGB color space; the color mapping process extracts the processed image color information and accurately maps it to the three-dimensional point cloud data, ensuring that the color information can accurately correspond to each point in the point cloud data; the color mapping process solves the color consistency problem between the underwater image and the point cloud data by avoiding excessive enhancement of noise while enhancing the local details of the image, thereby improving the visual effect of underwater three-dimensional reconstruction.

8. The underwater three-dimensional measurement method combining vision and line structured light according to claim 4, characterized in that: In step 2, the line structured light plane equation is obtained by extracting the intersection points between the center lines of the laser lines on multiple line structured light calibration images in the air and the circular intersection lines of the circular calibration plate, converting the two-dimensional points into three-dimensional points, and then fitting them using the least squares method.

9. The underwater three-dimensional measurement method combining vision and line structured light according to claim 2, characterized in that: In the step 1, when the line structured light image is preprocessed, it is first converted into a monochrome image and binarized, and the constrained binary grayscale value is 125. Then, the scattering noise near the laser line is reduced by setting the convolution kernel to (2, 2) through a corrosion operation, and then a dilation operation is performed to maintain the integrity of the laser line.

10. The underwater three-dimensional measurement method combining vision and line structured light according to claim 1, characterized in that: In step 3, the affine transformation is achieved by calculating 8 position parameters of the affine transformation matrix to restore the image taken underwater to the image coordinates in the air.

Images