Underwater three-dimensional point cloud generation method of line laser scanning device

Abstract

The invention discloses an underwater three-dimensional point cloud generation method of a line laser scanning device, and belongs to the field of laser. The galvanometer is used for refracting the laser emitted by the laser; the controller is used for controlling the deflection amplitude and the angular velocity of the galvanometer; and the camera is used for shooting the process of scanning and traversing the target by the laser rays refracted by the galvanometer. The method comprises the following steps of: calibrating the camera; carrying out distortion processing; extracting laser rays; calibrating a laser plane; three-dimensional reconstruction: obtaining three-dimensional point cloud data of the target surface; according to the method in the three-dimensional point cloud generation process, the three-dimensional point cloud generation of the underwater scene is realized based on the underwater double refraction model when the image is processed, and finally, the noise of the obtained target point cloud is removed to obtain the three-dimensional point cloud data with relatively high robustness.

Description

Technical Field

[0001] The invention belongs to the field of lasers and relates to a method for generating underwater three-dimensional point clouds of a line laser scanning device. Background Art

[0002] Currently, underwater robots are widely used in marine development and construction. Traditional two-dimensional imaging methods are limited by the physical properties of the underwater environment (such as light scattering and absorption) and the loss of depth information. These methods cannot meet the detailed environmental perception requirements required by underwater robots for close-range, precise tasks such as autonomously grasping seafood and inspecting and maintaining underwater facilities. Consequently, underwater three-dimensional imaging technology has garnered widespread attention.

[0003] Among the many underwater three-dimensional imaging technologies, underwater line laser scanning imaging has become one of the important methods for obtaining three-dimensional point clouds of small underwater scenes at close range due to its highly focused optical properties, high accuracy and resolution, and suitability for close-range precision operations.

[0004] Line laser scanning imaging technology uses an optical device to generate a highly focused beam line with a very small spot size, enabling extremely high spatial resolution within a tiny area. This characteristic is extremely advantageous for capturing detailed changes on an object's surface. A line laser is projected onto the surface of an object, forming a clear bright line pattern that captures the object's surface contours. The laser's monochromatic nature and high brightness easily distinguish it from its surroundings, making it highly resistant to interference in complex environments. Within a relatively small area, a line laser can cover and clearly scan the complex shapes and textures of an object. In close-range scenarios, the short focal length results in a sharper spot, which helps improve imaging accuracy.

[0005] Existing underwater three-dimensional point cloud generation methods have the following problems: the edge and color features of underwater images are not obvious, the feature point detection algorithm has difficulty in effectively extracting feature points, and the existing point cloud anomaly detection and optimization algorithms usually have high computational complexity and slow inference speed, making it difficult to meet real-time requirements. Summary of the Invention

[0006] In order to solve the above problems, the technical solution adopted by the present invention is: a method for generating underwater three-dimensional point clouds using a line laser scanning device, the line laser scanning device comprising: Laser: used to scan the target; Galvanometer: used for refracting the laser light emitted by the laser; Controller: controls the deflection amplitude and angular velocity of the galvanometer; Camera: photographing the process of the laser line refracted by the galvanometer scanning across the target, to obtain an image of the target scanning process; A sealed transparent shell is provided outside the laser, galvanometer, controller and camera; The underwater three-dimensional point cloud generation method includes the following steps: S1: Calibrate the internal and external parameters of the camera underwater to obtain the distortion coefficient and internal and external parameters of the camera underwater. The distortion coefficient corrects the geometric distortion caused by the camera lens. The obtained internal and external parameters of the camera are calibrated to establish a correspondence between the image pixel coordinates and the three-dimensional coordinates in space. S2: Processing the captured image during the laser scanning target process, extracting the region of interest or features in the image, and extracting the target laser stripe; The process of processing the captured image during the laser scanning target process, extracting the region of interest or features in the image, and extracting the target laser stripe is as follows: S21: Preprocessing the captured images during the laser scanning target process; The preprocessing adopts the underwater light transmission model, and the specific expression is as follows:

[0007] in: is the observed value of the pixel coordinates in the underwater image; is the true color of the point in the scene, is the transmission diagram, which shows the attenuation of light before it reaches the camera; is the attenuation coefficient of water, which is related to the wavelength; is the distance between the point and the camera; is the backscattered background light; S22: Enhance the image and simulate laser reflection, simulate the scattering and absorption process of the laser line passing through water, reversely restore the real laser energy, and perform color compensation or band reconstruction on the laser to achieve the effect of enhancing the experimental data; S23: In the region of interest extraction stage, an adaptive brightness and color space fusion extraction algorithm is proposed. Morphological directional filters are used to eliminate nonlinear interference, spatial consistency discrimination is introduced, and the geometric characteristics of laser lines are used to filter out pseudo lines. Based on the video acquisition characteristics, inter-frame maximum projection is used to highlight stable lasers, and laser trajectory is tracked to achieve robust extraction. S3: extract the center line of the extracted target laser strip; S4: Take two or more images of the laser line projected on the dot calibration plate in different postures, perform image preprocessing on the images, segment the laser strip, extract the center line, and obtain a two-dimensional coordinate array. Based on the internal and external parameters obtained by camera calibration, establish a correspondence between the image pixel coordinates and the three-dimensional coordinates in space to obtain a three-dimensional coordinate array; use the three-dimensional coordinate array as input, and fit the plane equation of the laser line through the least squares method to obtain the parameters of the plane equation; S5: After the parameters of the laser plane equation are determined in the initial state, the real-time parameters of the laser plane in each frame during the laser line rotation scanning target are obtained according to the rotation axis of the laser plane and the galvanometer rotation angular velocity controlled by the mainboard. The two-dimensional points extracted from the laser line of each frame are combined with the internal and external parameters of the camera to establish the correspondence between the image pixel coordinates and the three-dimensional coordinates in space, and the three-dimensional unit vector corresponding to the two-dimensional pixel coordinates is obtained. The three-dimensional unit vector and the laser plane equation are combined to obtain a unique three-dimensional point solution. The three-dimensional points of the entire line are generated in this way. Then, all laser line scanning target images are processed to obtain a global three-dimensional point cloud and realize the reconstruction of the three-dimensional point cloud. S6: Perform denoising and filtering on the global three-dimensional point cloud to obtain the final point cloud.

[0008] Furthermore, the process of performing denoising filtering on the obtained global three-dimensional point cloud to obtain the final generated point cloud is as follows: S61: Construct a high-quality point cloud mapping image, project the original 3D point cloud onto a 2D plane, and achieve 3D geometry-2D visual information fusion; S62: Use the segmentation model to perform pixel-level segmentation on the mapped image to locate potential noise areas; S63: Performing region screening based on the segmentation results, and performing secondary discrimination on the suspected noise points by combining multi-scale context information and boundary consistency to obtain the noise region; S64: Map the noise area back to the three-dimensional space and remove it, so as to perform denoising filtering on the global three-dimensional point cloud and obtain the final point cloud.

[0009] Furthermore: the line laser scanning device also includes a galvanometer support structure for placing the galvanometer, and a sliding module for sliding the galvanometer support structure.

[0010] Furthermore, the process of calibrating the internal and external parameters of the camera underwater to obtain the distortion coefficient and internal and external parameters of the camera underwater is as follows: The imaging point of the underwater target Pw on the image plane is , are the horizontal and vertical coordinates of the pixel points respectively; if the imaging point is in the pinhole refraction model, the direct imaging point is , underwater, after multiple refractions, the pinhole refraction model no longer holds; if refraction compensation is not performed, the obtained The point is not an actual point, that is, a virtual point and There is a relationship between:

[0011] in: is the vertical distance from the laser emission point to the imaging plane, and θ8 is the angle between the erroneous light transmission path corresponding to the virtual point and the horizontal direction; Substitute the refraction compensation parameters into the camera calibration program, extract the dots from the photographed dot calibration plate, and after refraction compensation, obtain the real underwater dot pixel coordinate array. Then use these coordinate data for calibration to obtain the camera's distortion coefficient and internal and external parameters underwater. Underwater optical distortion correction: Correct the geometric distortion caused by the camera lens using the distortion coefficients obtained through calibration; Refraction compensation: An underwater double refraction model is established. Using this model, the true direction of light underwater is inferred from the image coordinates captured by the camera, and then remapped to the corrected true image coordinates.

[0012] Furthermore, the process of extracting the center line of the extracted target laser strip is as follows: The mean square grayscale gradient method is used to obtain the normal direction of the target laser bar at each point on the target laser bar area, and the weighted grayscale centroid method is used along the normal direction of the target laser bar to obtain the sub-pixel center initial point of the target laser bar. Finally, a piecewise cubic polynomial curve fitting is performed on the initial point of the sub-pixel center of the target laser stripe to obtain the sub-pixel center coordinates of the final light stripe, completing the centerline extraction of the target laser stripe.

[0013] Furthermore, the formula used to calibrate the internal and external parameters of the camera underwater is as follows:

[0014] in: Is the value of the point on the target object in the Z-axis direction in the camera coordinate system, ( , ) is the pixel coordinate of the point on the target object on the image, and Indicates the pixel size of the camera, :For along The physical size corresponding to each pixel in the direction; :For along The physical size of each pixel in the direction, ( , ) is the coordinate of the point in the pixel coordinate system, is the camera focal length, R is the rotation matrix, T is the displacement matrix, ( , , ) is the coordinate of the point in the world coordinate system, and are the horizontal and vertical coordinates of the camera's focal length respectively.

[0015] The present invention provides a method for generating underwater three-dimensional point clouds using a line laser scanning device. The method uses a galvanometer laser to scan a turbid underwater environment while simultaneously capturing images with an underwater camera. The obtained image data is processed by the method to generate a three-dimensional point cloud map of the underwater scene surface. Point cloud images can be used by underwater fishing robots to estimate the location and size of seafood and provide three-dimensional coordinates to assist in fishing. A new underwater camera calibration model for underwater image correction uses an approximate single-view camera model to correct for water vapor refraction distortion and lens distortion respectively. Combining optical cameras with structured light, and leveraging the camera's optical capture and the laser's minimal impact in underwater environments, it generates a wide-area, high-precision 3D point cloud. This model has the following advantages: 1. The camera parameters after calibration are reverse mapped, and the pixel mean square error is only 0.4 pixels.

[0016] 2. It can generate point clouds with high robustness and high speed.

[0017] 3. Able to precisely control the rotation of the galvanometer; 4. The assembled slide and galvanometer support parts are designed and 3D printed by ourselves, with very low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 It is a structural diagram of a line laser scanning device; Figure 2 It is a line laser scanning device without a sealed transparent housing; Figure 3 It is the effect diagram of enhancing experimental data; Figure 4 It is to track the laser trajectory to achieve robust extraction map; Figure 5A perspective view of the centerline of the extracted laser stripe; Figure 6 A close-up view of the centerline of the extracted laser stripe; Figure 7 Schematic diagram of line laser scanning imaging; Figure 8 This is a diagram of the denoising and filtering process of underwater 3D point clouds; Figure 9 This is the image before denoising (the red box is the target area); Figure 10 is the denoised image; Figure 11 It is a refraction model of multiple media of air-waterproof shell-water; Figure 12 It is the pose graph shot by the calibration plate; Figure 13 is the calibration error analysis diagram; Figure numerals: 1. waterproof cabin shell, 2. camera module, 3. line laser, 4. bottom plate, 5. slide groove, 6. galvanometer, 7. galvanometer support, 8. waterproof cabin cover. DETAILED DESCRIPTION

[0020] It should be noted that, unless there is any conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] Figure 1 It is a structural diagram of a line laser scanning device; Figure 2 It is a line laser scanning device without a sealed transparent housing; A line laser scanning device, comprising: Laser 3; used for scanning the target; Galvanometer 6: used to refract the laser light emitted by the laser 3; Controller: controls the deflection amplitude and angular velocity of the galvanometer 6; and a supporting member 7 provided below the galvanometer 6; Camera 2: captures the laser line refracted by the galvanometer 6 as it scans across the target. Using a low-cost camera 2 facilitates its popularization and application in engineering projects. A sealed transparent shell 1 is provided outside the line laser scanning device: The sealed transparent housing 1 is arranged outside the point cloud generation module; the sealed transparent housing 1 can be used in underwater scenes; the sealed transparent housing 1 can be made of glass; the sealed transparent housing 1 includes a housing arranged on all sides, and waterproof hatches 8 arranged above and below; A bottom plate 4 is also provided below the line laser scanning device; Furthermore, it also includes a galvanometer support structure for placing the galvanometer 6.

[0023] Furthermore, it also includes a sliding module for sliding the galvanometer support structure. The sliding module is a rectangular parallelepiped structure with a sliding groove provided on the rectangular parallelepiped structure; the galvanometer support structure is fixed to the sliding groove 5 by screws; the sliding groove 5 is a hollow rounded rectangle; The camera 2 is arranged above the laser 3 .

[0024] The code consists of the galvanometer 6 control code, camera 2 calibration, laser line extraction, laser plane calibration, three-dimensional reconstruction, camera 2 distortion, and double refraction. That is, the digital signal is output through the Arduino motherboard, and then the digital-to-analog conversion module DAC converts the digital signal into a voltage signal. After passing through the driver board and power board, the galvanometer 6 is driven to rotate. The required control code is written to control the rotation of the galvanometer 6.

[0025] A method for generating an underwater three-dimensional point cloud using a line laser scanning device comprises the following steps: S1: Calibrate the camera's internal and external parameters underwater based on the underwater double refraction model of light, obtaining the camera's distortion coefficients and internal and external parameters underwater. The distortion coefficients can correct for the geometric distortion caused by the camera lens. The camera's internal and external parameters obtained by calibration can establish a correspondence between image pixel coordinates and three-dimensional coordinates in space. S2: Processing the captured image during the laser scanning target process, extracting the region of interest or features in the image, and extracting the target laser stripe; The process of processing the captured image during the laser scanning target process, extracting the region of interest or features in the image, and extracting the target laser stripe is as follows: S21: Preprocessing the captured images during the laser scanning target process; The preprocessing adopts the underwater light transmission model, and the specific expression is as follows: (1) in: is the observed value of the pixel coordinates in the underwater image; is the true color of the point in the scene, is the transmission diagram, which shows the attenuation of light before it reaches the camera; is the attenuation coefficient of water, which is related to the wavelength; is the distance between the point and the camera; is the backscattered background light; S22: Enhance the image and simulate the laser reflection, simulate the scattering and absorption process of the laser line passing through water, reversely restore the real laser energy, and perform color compensation or band reconstruction on the laser to achieve the effect of enhancing the experimental data, such as Figure 3 As shown; S23: In the region of interest extraction stage, an adaptive brightness + color space fusion extraction algorithm is proposed, which uses morphological directional filters to eliminate nonlinear interference, introduces spatial consistency discrimination, uses the geometric characteristics of laser lines to filter pseudo lines, and uses inter-frame maximum projection to highlight stable lasers based on video acquisition characteristics, and tracks laser trajectories to achieve robust extraction; Figure 4 As shown; S3: Extracting the center line of the laser strip: extracting the center line of the extracted laser strip; Figure 5 A perspective view of the centerline of the extracted laser stripe; Figure 6 A close-up view of the centerline of the extracted laser stripe; S4: laser plane calibration; Take two or more images of the laser line projected onto the dot calibration plate in different positions, perform the aforementioned image preprocessing on the images, segment the laser strip, extract the centerline, and obtain a two-dimensional coordinate array. Based on the internal and external parameters obtained from camera calibration, establish a correspondence between the image pixel coordinates and the three-dimensional coordinates in space to obtain a three-dimensional coordinate array; use the three-dimensional coordinate array as input and fit the plane equation of the laser line using the least squares method to obtain the parameters of the plane equation; S5: 3D reconstruction: After the parameters of the laser plane equation are determined in the initial state, the real-time parameters of each frame of the laser plane during the laser line rotation scanning target are obtained based on the rotation axis of the laser plane and the galvanometer rotation angular velocity controlled by the mainboard. The two-dimensional points extracted from each frame of the laser line are combined with the internal and external parameters of the camera to establish the correspondence between the image pixel coordinates and the three-dimensional coordinates in space, and obtain the three-dimensional unit vector corresponding to the two-dimensional pixel coordinates. The three-dimensional unit vector and the laser plane equation are combined to obtain a unique three-dimensional point solution. The three-dimensional points of the entire line are generated in this way, and then all laser line scanning target images are processed to obtain a global three-dimensional point cloud and realize the reconstruction of the three-dimensional point cloud. The expression of the laser space plane is as follows: (2) , , is the coordinate value of a certain point of the target in the camera coordinate system; A, B, C, D are plane coefficients; The parametric equation of a line in space is as follows: (3) S is the conversion unit factor.

[0026] Figure 7 It is a schematic diagram of line laser scanning imaging; This application introduces two-dimensional image semantic information to assist in three-dimensional point cloud noise removal, achieving the synergy of cross-dimensional information fusion and intelligent denoising.

[0027] S6: Underwater 3D point cloud denoising filtering: Filtering the global 3D point cloud to obtain the final point cloud; The process of performing denoising and filtering on the global three-dimensional point cloud to obtain the final point cloud is as follows: S61: Construct a high-quality point cloud mapping image, project the original 3D point cloud onto a 2D plane, and achieve 3D geometry-2D visual information fusion; S62: Use the SAM (Segment Anything Model) segmentation model to perform pixel-level segmentation on the mapped image and accurately locate potential noise areas; S63: Performing region screening based on the segmentation results, combining multi-scale context information and boundary consistency to perform secondary discrimination on suspected noise points to obtain noise regions, thereby improving the reliability of noise recognition; S64: Map the noise area back to the three-dimensional space and remove it, thereby achieving efficient and high-precision point cloud denoising, and performing denoising filtering on the global three-dimensional point cloud to obtain the final generated point cloud.

[0028] Figure 8 This is a diagram of the denoising and filtering process of underwater 3D point clouds; Figure 9 This is the image before denoising (the red box is the target area); Figure 10 is the denoised distant image; Furthermore, the process of calibrating the internal and external parameters of the camera underwater based on the double refraction model of light underwater and obtaining the distortion coefficient and internal and external parameters of the camera underwater is as follows: When working underwater, the instrument is sealed in a waterproof device, so the underwater target and the laser scanning system are isolated in media with different refractive indices. When the camera shoots underwater objects, the light passes through the horizontal glass waterproof cover and the interface between the glass waterproof cover and the air. After refraction twice, the object is finally imaged on the image surface of the camera. There is a refraction error when acquiring optical data. Establishing an effective refraction model for refraction compensation is of significant significance to improving the accuracy of the overall system. Therefore, taking into account the refractive index and Snell's law between multiple media, a refraction model of multiple media of air-waterproof housing-water is established, such as Figure 11 As shown; S11: Calibrate camera 2; obtain calibration plate images at different angles, and calibrate the intrinsic and extrinsic parameters of camera 2 using the camera 2 calibration method; the process of calibrating the intrinsic and extrinsic parameters of the camera using the camera calibration method is as follows: Depend on Figure 3 It can be seen that the imaging point of the underwater target Pw on the image plane is , if it is in the pinhole refraction model, the direct imaging point is Underwater, after multiple refractions, the pinhole refraction model no longer holds. Without refraction compensation, the obtained The point is not an actual point, that is, a virtual point and There is a relationship between:

[0029] in: is the vertical distance from the laser emission point to the imaging plane, and θ8 is the angle between the erroneous light transmission path corresponding to the virtual point and the horizontal direction; Substitute the refraction compensation parameters into the camera calibration program, and calibrate the points on the dot calibration plate at multiple angles. Figure 12 , extract the dot's 2D coordinates, and after refraction compensation, obtain the real underwater dot pixel coordinate array. Then use these coordinate data for calibration to obtain the camera's underwater distortion coefficient and internal and external parameters. The calibration result is relatively accurate, and the mean square error of the remapping result is only 0.4 pixels. Figure 13 As shown; S12: Underwater optical distortion correction: The geometric distortion caused by the camera lens is corrected by the distortion coefficient obtained through calibration. Furthermore, the calibration method of the camera 2 of the present application is simple and efficient, does not require expensive precision calibration equipment, and is simple and low-cost. By obtaining checkerboard images at different angles through multiple photos, the least squares method is used to optimize the intrinsic and extrinsic parameters to achieve high calibration accuracy. It has strong robustness against slight deformation and noise of the checkerboard pattern and can overcome the problems of image noise and slight deformation of the checkerboard to a certain extent. It takes into account both computational efficiency and accuracy.

[0030] The formula used to calibrate the camera 2 and the intrinsic and extrinsic parameters using the least squares method is as follows: (5) in: is the value of the point in the Z-axis direction in the camera 2 coordinate system, (u, v) is the pixel coordinate of the point on the image, and represents the pixel size of camera 2, :along The physical size of each pixel in the direction, :along The physical size of each pixel in the direction. , ) is the coordinate of the point in the pixel coordinate system, is the focal length of camera 2. R is the rotation matrix, T is the translation matrix, ( , , ) is the coordinate of the point in the world coordinate system, = / , = / , multiply the two matrices on the right side of the first equation and simplify it; and is the focal length of the camera, usually measured in pixels. They are related to the physical focal length of the camera sensor (usually measured in millimeters) and the pixel size of the image (i.e., how many millimeters each pixel represents).

[0031] Furthermore, by grayscale processing the image and calculating the weighted center of the pixel grayscale within the target area of ​​the captured laser line, sub-pixel accuracy can be achieved. Compared with position judgment based solely on pixels, the accuracy is significantly improved, making it suitable for scenarios requiring high positioning accuracy. To a certain extent, it is robust to changes in image illumination, noise, and local distortion, because the line laser centerline extraction method utilizes the overall information of the regional grayscale value rather than the grayscale value of a single point or line. The grayscale centroid method has a simple algorithm and only requires calculating the weighted average, making it suitable for real-time processing. Compared with complex feature extraction algorithms, its calculation speed is faster, making it suitable for embedded systems or real-time applications. The calculation process is relatively simple and easy to implement. It does not require complex parameter adjustments or large amounts of data training, making it very suitable for application in lightweight environments of underwater robots.

[0032] Furthermore, the process of extracting the center line of the extracted target laser strip is as follows: Enhance the image and simulate the laser reflection. Simulate the scattering and absorption process of the laser line through water, reversely restore the real laser energy, and perform color compensation or band reconstruction on the laser to achieve the effect of enhancing the experimental data (such as Figure 3 ). In the region of interest extraction stage, an adaptive brightness + color space fusion extraction algorithm is proposed, which uses morphological directional filters to eliminate nonlinear interference, introduces spatial consistency discrimination, and uses the geometric characteristics of laser lines to filter pseudo lines. According to the video acquisition characteristics, the inter-frame maximum projection is used to highlight the stable laser, and the laser trajectory is tracked to achieve robust extraction of the target laser bar area. The mean square grayscale gradient method is used to calculate the normal direction of the target laser bar at each point on the target laser bar skeleton, and the weighted grayscale centroid method is used along the normal direction of the target laser bar to obtain the sub-pixel center initial point of the target laser bar. Finally, a piecewise cubic polynomial curve fitting is performed on the initial point of the sub-pixel center of the target laser stripe to obtain the sub-pixel center coordinates of the final light stripe, completing the centerline extraction of the target laser stripe.

[0033] The formula for extracting the center line of the laser bar is as follows: (6) in: :It is the center line point of the laser bar The horizontal axis, : Weight factor, used to indicate the weight of different points in the calculation; :point of Coordinate values ​​indicate the specific locations of the points participating in weighted averaging.

[0034] : Indicates the number of rows in the image, usually along the vertical direction ( axis). In a two-dimensional image, " " is a horizontal line from top to bottom, that is, the coordinates from row 1 to row n. Usually in image processing, It represents the vertical coordinate of the image.

[0035] : Indicates the number of columns of the image, usually along the horizontal direction ( axis). In the image, " " is the vertical line from left to right, that is, the coordinates from column 1 to column n. It represents the horizontal coordinate of the image.

[0036] Here we represent the midpoint of the image ( , ) brightness value. ∑: represents the sum of all points involved in the calculation.

[0037] Furthermore, the formula used for underwater optical distortion correction is as follows: (7) The above formula (7) is transformed into: (8) This formula describes the correction model for lens distortion, specifically radial and tangential distortion in optical systems. Radial distortion: The image center distorts outward or inward, commonly seen in barrel distortion and pincushion distortion. Tangential distortion: The image pixels shift and tilt due to incomplete alignment between the lens and the sensor.

[0038] in: , : Represents the horizontal and vertical coordinates of the original image (uncorrected pixel coordinates). : Represents the image coordinates after distortion correction (pixel coordinates after distortion is eliminated). , , : represent radial distortion coefficients. They describe the radial distortion in the image caused by the lens (such as "barrel distortion" or "pincushion distortion"); radial distortion increases with the radius r from the center; , : Represents the tangential distortion coefficients. They describe the tangential distortion caused by imperfect lens alignment, where some points in the image appear offset horizontally or vertically. r: The radius from the point to the image center, calculated as: r² = x² + y².

[0039] Furthermore: the formula for fitting the laser line plane is as follows: (9) Formula (7) is converted into the following formula: (10) in:( , , ) is the coordinate value, , , is the plane parameter to be determined. S is the square of the value of all points to the fitting plane to be determined. The formula is ( , , ) to find the minimum value of S corresponding to ( , , ) is the coefficient of the most suitable laser plane. The formula on the left is to find the minimum value of S. The formula on the right is obtained by differentiation. There are a total of n groups of three-dimensional points, and I refers to a specific group among them.

[0040] Considering the characteristics of the underwater environment, the camera is always placed in a waterproof housing during image data acquisition and preprocessing. Therefore, additional distortion caused by refraction between the water, the waterproof housing, and the air is introduced into the system. When calibrating underwater Camera 2, traditional calibration methods treat water-air refraction distortion as lens distortion, which introduces additional errors. Therefore, we designed a new underwater image correction model for Camera 2 (combining a Camera 2 calibration method with a birefringence model). This model uses an approximate single-view Camera 2 model to separately correct for water-air refraction distortion and lens distortion. By calibrating Camera 2's intrinsic and extrinsic parameters, refraction distortion is corrected by establishing an underwater birefringence model. Assume that a ray originates from a 3D point and intersects the image plane at the image point. The distance between the waterproof glass and the Camera 2 lens is denoted by d. The thickness of the waterproof housing is much smaller than the world point depth. Therefore, the thickness of the waterproof housing can be assumed to be zero. Therefore, in the proposed underwater Camera 2 calibration model, refraction should only occur once between water and air. Furthermore, the optical axis is assumed to be perpendicular to the water-air interface. According to Snell's law, refraction between water and air can be quantitatively described. By creating two virtual imaging planes, we can eliminate the refraction of light caused by the different refractive indices of water and air during camera 2 imaging. Ultimately, we achieve an imaging model that approximates that of a single-view camera 2. This approach also allows for separate corrections for lens distortion and distortion caused by underwater-air refraction, achieving higher accuracy.

[0041] Example 1: First, take about 20 pictures of the dot calibration plate underwater to calibrate the parameters of camera 2.

[0042] Modeling of underwater camera 2 with air-water-waterproof housing refraction.

[0043] Use an Arduino Nano to control a galvanometer laser line to scan the target scene, and a camera to capture the scanned data. Fit the laser plane, calculate the parameters of each frame's laser plane based on the angular velocity, extract the unit vectors of three points on the laser line, and solve them with the plane equation to determine the coordinates of the three-dimensional points.

[0044] And so on, the entire scene model is traversed to generate a point cloud.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating underwater three-dimensional point clouds using a line laser scanning device, characterized in that: Line laser scanning device, comprising: Laser: used to scan the target; Galvanometer: used for refracting the laser light emitted by the laser; Controller: controls the deflection amplitude and angular velocity of the galvanometer; Camera: photographing the process of the laser line refracted by the galvanometer scanning across the target, to obtain an image of the target scanning process; A sealed transparent shell is provided outside the laser, galvanometer, controller and camera; The underwater three-dimensional point cloud generation method includes the following steps: S1: Calibrate the internal and external parameters of the camera underwater to obtain the distortion coefficient and internal and external parameters of the camera underwater. The distortion coefficient corrects the geometric distortion caused by the camera lens. The obtained internal and external parameters of the camera are calibrated to establish a correspondence between the image pixel coordinates and the three-dimensional coordinates in space. S2: Processing the captured image during the laser scanning target process, extracting the region of interest or features in the image, and extracting the target laser stripe; The process of processing the captured image during the laser scanning target process, extracting the region of interest or features in the image, and extracting the target laser stripe is as follows: S21: Preprocessing the captured images during the laser scanning target process; The preprocessing adopts the underwater light transmission model, and the specific expression is as follows: in: is the observed value of the pixel coordinates in the underwater image; is the true color of the point in the scene, is the transmission diagram, which shows the attenuation of light before it reaches the camera; is the attenuation coefficient of water, which is related to the wavelength; is the distance between the point and the camera; is the backscattered background light; S22: Enhance the image and simulate laser reflection, simulate the scattering and absorption process of the laser line passing through water, reversely restore the real laser energy, and perform color compensation or band reconstruction on the laser to achieve the effect of enhancing the experimental data; S23: In the region of interest extraction stage, an adaptive brightness and color space fusion extraction algorithm is proposed. Morphological directional filters are used to eliminate nonlinear interference, spatial consistency discrimination is introduced, and the geometric characteristics of laser lines are used to filter out pseudo lines. Based on the video acquisition characteristics, inter-frame maximum projection is used to highlight stable lasers, and laser trajectory is tracked to achieve robust extraction. S3: extract the center line of the extracted target laser strip; S4: Take two or more images of the laser line projected on the dot calibration plate in different postures, perform image preprocessing on the images, segment the laser strip, extract the center line, and obtain a two-dimensional coordinate array. Based on the internal and external parameters obtained by camera calibration, establish a correspondence between the image pixel coordinates and the three-dimensional coordinates in space to obtain a three-dimensional coordinate array; use the three-dimensional coordinate array as input, and fit the plane equation of the laser line through the least squares method to obtain the parameters of the plane equation; S5: After the parameters of the laser plane equation are determined in the initial state, the real-time parameters of the laser plane in each frame during the laser line rotation scanning target are obtained according to the rotation axis of the laser plane and the galvanometer rotation angular velocity controlled by the mainboard. The two-dimensional points extracted from the laser line of each frame are combined with the internal and external parameters of the camera to establish the correspondence between the image pixel coordinates and the three-dimensional coordinates in space, and the three-dimensional unit vector corresponding to the two-dimensional pixel coordinates is obtained. The three-dimensional unit vector and the laser plane equation are combined to obtain a unique three-dimensional point solution. The three-dimensional points of the entire line are generated in this way. Then, all laser line scanning target images are processed to obtain a global three-dimensional point cloud and realize the reconstruction of the three-dimensional point cloud. S6: Perform denoising and filtering on the global three-dimensional point cloud to obtain the final point cloud.

2. The method for generating underwater three-dimensional point clouds using a line laser scanning device according to claim 1, wherein: The process of performing denoising and filtering on the global three-dimensional point cloud to obtain the final point cloud is as follows: S61: Construct a high-quality point cloud mapping image, project the original 3D point cloud onto a 2D plane, and achieve 3D geometry-2D visual information fusion; S62: Use the segmentation model to perform pixel-level segmentation on the mapped image to locate potential noise areas; S63: Performing region screening based on the segmentation results, and performing secondary discrimination on the suspected noise points by combining multi-scale context information and boundary consistency to obtain the noise region; S64: Map the noise area back to the three-dimensional space and remove it, so as to perform denoising filtering on the global three-dimensional point cloud and obtain the final point cloud.

3. The method for generating underwater three-dimensional point clouds using a line laser scanning device according to claim 1, wherein: The line laser scanning device also includes a galvanometer support structure for placing the galvanometer, and a sliding module for sliding the galvanometer support structure.

4. The method for generating underwater three-dimensional point clouds using a line laser scanning device according to claim 1, wherein: The process of calibrating the internal and external parameters of the camera underwater to obtain the distortion coefficient and internal and external parameters of the camera underwater is as follows: The imaging point of the underwater target Pw on the image plane is , are the horizontal and vertical coordinates of the pixel points respectively; if the imaging point is in the pinhole refraction model, the direct imaging point is , underwater, after multiple refractions, the pinhole refraction model no longer holds; if refraction compensation is not performed, the obtained The point is not an actual point, that is, a virtual point and There is a relationship between: in: is the vertical distance from the laser emission point to the imaging plane, and θ8 is the angle between the erroneous light transmission path corresponding to the virtual point and the horizontal direction; Substitute the refraction compensation parameters into the camera calibration program, extract the dots from the photographed dot calibration plate, and after refraction compensation, obtain the real underwater dot pixel coordinate array. Then use these coordinate data for calibration to obtain the camera's distortion coefficient and internal and external parameters underwater. Underwater optical distortion correction: Correct the geometric distortion caused by the camera lens using the distortion coefficients obtained through calibration; Refraction compensation: An underwater double refraction model is established. Using this model, the true direction of light underwater is inferred from the image coordinates captured by the camera, and then remapped to the corrected true image coordinates.

5. The method for generating underwater three-dimensional point clouds using a line laser scanning device according to claim 1, wherein: The process of extracting the center line of the extracted target laser strip is as follows: The mean square grayscale gradient method is used to obtain the normal direction of the target laser bar at each point on the target laser bar area, and the weighted grayscale centroid method is used along the normal direction of the target laser bar to obtain the sub-pixel center initial point of the target laser bar. Finally, a piecewise cubic polynomial curve fitting is performed on the initial point of the sub-pixel center of the target laser stripe to obtain the sub-pixel center coordinates of the final light stripe, completing the centerline extraction of the target laser stripe.

6. The method for generating underwater three-dimensional point clouds using a line laser scanning device according to claim 1, wherein: The formula used to calibrate the internal and external parameters of the camera underwater is as follows: in: Is the value of the point on the target object in the Z-axis direction in the camera coordinate system, ( , ) is the pixel coordinate of the point on the target object on the image, and Indicates the pixel size of the camera, :For along The physical size corresponding to each pixel in the direction; :For along The physical size of each pixel in the direction, ( , ) is the coordinate of the point in the pixel coordinate system, is the camera focal length, R is the rotation matrix, T is the displacement matrix, ( , , ) is the coordinate of the point in the world coordinate system, and are the horizontal and vertical coordinates of the camera's focal length respectively.

Images