An underwater three-dimensional point cloud generation method of a linear laser scanning device

Abstract

The application discloses a kind of underwater three-dimensional point cloud generation methods of linear laser scanning device, belong to laser field, including laser;Galvanometer: for the laser that laser sends out refracts;Controller: the deflection amplitude and angular velocity of galvanometer are controlled;Camera: the process that laser line scanning traverses target is photographed by galvanometer refraction, the following steps of the method are: camera is calibrated;Distortion processing;Laser line extraction;Laser plane calibration;Three-dimensional reconstruction: obtain the three-dimensional point cloud data of target surface;The method in the three-dimensional point cloud generation process of above is based on underwater double refraction model when image is handled, realize the three-dimensional point cloud generation of underwater scene, finally again to the target point cloud obtained eliminates noise, obtains the three-dimensional point cloud data of higher robustness.

Description

Technical Field

[0001] This invention belongs to the field of lasers and relates to a method for generating underwater three-dimensional point clouds using a line laser scanning device. Background Technology

[0002] Currently, underwater robots are widely used in marine development and construction. However, traditional two-dimensional imaging methods are limited by the physical characteristics of the underwater environment (light scattering and absorption, etc.) and the loss of depth information, failing to meet the needs of underwater robots for detailed environmental perception when performing close-range, precision tasks such as autonomously grasping marine products and inspecting and maintaining underwater facilities. Therefore, underwater three-dimensional imaging technology has attracted widespread attention.

[0003] Among numerous underwater 3D imaging technologies, underwater line laser scanning imaging has become one of the important methods for acquiring 3D point clouds of small underwater scenes at close range due to its highly focused optical characteristics, high precision and resolution, and suitability for close-range precision operations.

[0004] Line laser scanning imaging technology generates a highly focused beam of light through optical devices. Its extremely small spot size allows for very high spatial resolution within a tiny area. This characteristic is highly advantageous for capturing detailed variations on an object's surface. By projecting a line laser onto the object's surface, a clear pattern of bright lines is formed, capturing the object's surface contours. The monochromaticity and high brightness of the laser easily distinguish it from the environment, giving it strong resistance to interference in complex environments. Within a small area, line lasers can cover and clearly scan the complex shapes and textures of objects. In close-range scenes, the short focal length and sharper spot contribute to improved imaging accuracy.

[0005] Existing methods for generating underwater 3D point clouds suffer from several drawbacks. These include unclear edge and color features in underwater images, difficulty in effectively extracting feature points using feature point detection algorithms, and high computational complexity and slow inference speed in existing point cloud anomaly detection and optimization algorithms, making it difficult to meet real-time requirements. Summary of the Invention

[0006] To solve the above problems, the technical solution adopted by the present invention is: an underwater three-dimensional point cloud generation method using a line laser scanning device, the line laser scanning device comprising:

[0007] Laser: Used for scanning targets;

[0008] Galvanometer: Used to refract the laser light emitted by the laser.

[0009] Controller: Controls the deflection amplitude and angular velocity of the galvanometer;

[0010] Camera: take a picture of the process of the galvanometer refracted laser line scanning the target, to obtain the scanned target process picture image;

[0011] A sealed transparent shell is arranged outside the laser, galvanometer, controller and camera;

[0012] The underwater three-dimensional point cloud generation method comprises the following steps:

[0013] S1: calibrate the internal and external parameters of the camera under water, obtain the distortion coefficient and internal and external parameters of the camera under water, correct the geometric distortion caused by the camera lens by the distortion coefficient, obtain the internal and external parameters of the camera by calibration, and establish the corresponding relationship between the image pixel coordinates and the three-dimensional coordinates in space;

[0014] S2: process the picture image of the laser scanning target process, extract the region of interest or feature in the image, and perform target laser strip extraction;

[0015] The process of processing the picture image of the laser scanning target process, extracting the region of interest or feature in the image, and performing target laser strip extraction is as follows:

[0016] S21: pre-process the picture image of the laser scanning target process;

[0017] The pre-processing adopts an underwater light transmission model, and the specific expression is as follows:

[0018]

[0019] Wherein: is the observed value of the pixel point coordinate in the underwater image;

[0020] is the real color of the point in the scene, is the transmission diagram, indicating the attenuation of light before reaching the camera;

[0021] is the attenuation coefficient of the water body, which is related to the wavelength; is the distance of the point from the camera; is the backscattered background light;

[0022] S22: enhance and simulate laser reflection of the image, simulate the scattering and absorption process of the laser line through the water, inversely restore the real laser energy, color compensate or waveband reconstruct the laser, and achieve the effect of enhancing the experimental data;

[0023] S23: In the region of interest extraction stage, an adaptive brightness and color space fusion extraction algorithm is proposed, a morphological directional filter is used to exclude nonlinear interference, spatial consistency discrimination is introduced, the geometric characteristics of the laser line are used to filter false lines, according to the video acquisition characteristics, the stable laser is highlighted by using the inter-frame maximum value projection, and the laser trajectory is tracked to realize robust extraction;

[0024] S3: Center line extraction is performed on the extracted target laser strip;

[0025] S4: More than two images of the laser line projected on the circle point calibration board in different pose states are photographed, image preprocessing is performed on the images, the laser strip is segmented, center line extraction is performed, a two-dimensional coordinate array is obtained, the corresponding relationship between the image pixel coordinates and the three-dimensional coordinates in the space is established according to the internal and external parameters obtained by camera calibration, and a three-dimensional coordinate array is obtained; the three-dimensional coordinate array is used as input, and the plane equation of the laser line is fitted by the least square method, and the parameters of the plane equation are obtained;

[0026] S5: After the parameters of the laser plane equation are determined in the initial state, the real-time parameters of each frame of the laser plane in the process of rotating scanning of the laser line are obtained according to the rotating axis of the laser plane and the rotating angular velocity of the galvanometer controlled by the mainboard, the two-dimensional point after extraction of each frame of the laser line is combined with the internal and external parameters of the camera, the corresponding relationship between the image pixel coordinates and the three-dimensional coordinates in the space is established, the three-dimensional unit vector corresponding to the two-dimensional pixel coordinates is obtained, the three-dimensional unit vector and the laser plane equation are combined to obtain a unique three-dimensional point solution, and the three-dimensional points of the whole line are generated in turn, and then all the laser line scanning target images are processed, so that the global three-dimensional point cloud is obtained, and the reconstruction of the three-dimensional point cloud is realized;

[0027] S6: The global three-dimensional point cloud is denoised and filtered to obtain the finally generated point cloud.

[0028] Further, the process of denoising and filtering the global three-dimensional point cloud to obtain the finally generated point cloud is as follows:

[0029] S61: A high-quality point cloud mapping image is constructed, the original three-dimensional point cloud is projected onto a two-dimensional plane, and three-dimensional geometry-two-dimensional vision information fusion is realized;

[0030] S62: A cut everything model is used to perform pixel-level segmentation on the mapping image, and a potential noise area is located;

[0031] S63: Based on the segmentation result, region screening is performed, suspected noise points are discriminated again in combination with multi-scale context information and boundary consistency, and a noise area is obtained;

[0032] S64: The noise area is mapped back to the three-dimensional space and removed, the global three-dimensional point cloud is denoised and filtered, and the finally generated point cloud is obtained.

[0033] Further, the linear laser scanning device further comprises a vibrating mirror support structure for placing the vibrating mirror, and a sliding module for sliding the vibrating mirror support structure.

[0034] Further, the process of calibrating the internal and external parameters of the camera under water to obtain the distortion coefficient and the internal and external parameters of the camera under water is as follows:

[0035] The imaging point of the underwater target Pw on the image plane is , The horizontal and vertical coordinates of the pixel point are respectively If the imaging point is in the pinhole refraction model, the direct imaging point is Under water, after multiple refractions, the pinhole refraction model no longer holds; if no refraction compensation processing is performed, the obtained point is not the actual real point, i.e., a virtual point and There is a relationship between the virtual point and the real point:

[0036]

[0037] Wherein: is the vertical distance from the laser emitting point to the imaging plane, and θ8 is the included angle between the incorrect light transmission route corresponding to the virtual point and the horizontal direction;

[0038] The parameters of the refraction compensation are substituted into the camera calibration program, the circular points in the circular point calibration plate are extracted, the two-dimensional coordinates of the circular points are subjected to refraction compensation to obtain an array of underwater real circular point pixel coordinates, and the coordinates are used for calibration to obtain the distortion coefficient and the internal and external parameters of the camera under water;

[0039] Underwater optical distortion correction: the distortion coefficient obtained through calibration is used to correct the geometric distortion caused by the camera lens;

[0040] Refraction compensation: an underwater double refraction model is established, the real direction of the light under water is calculated from the image coordinates captured by the camera by using the model, and the real image coordinates after correction are mapped.

[0041] Further, the process of extracting the center line of the extracted target laser strip is as follows:

[0042] The normal direction of the target laser strip at each point in the target laser strip region is obtained by using the mean square gray gradient method, and the initial point of the sub-pixel center of the target laser strip is obtained by using the weighted gray gravity center method along the normal direction of the target laser strip,

[0043] Finally, piecewise cubic polynomial curve fitting is performed on the initial sub-pixel center of the target laser stripe to obtain the final sub-pixel center coordinates of the light stripe, thus completing the extraction of the centerline of the target laser stripe.

[0044] Furthermore, the formula used to calibrate the camera's intrinsic and extrinsic parameters underwater is as follows:

[0045]

[0046] in: It is the value of a point on the target object in the Z-axis direction in the camera coordinate system. , () represents the pixel coordinates of a point on the target object in the image. and Indicates the camera's pixel size. : for along The physical size corresponding to each pixel in the orientation; : for along The physical size corresponding to each pixel in the direction, ( , () represents the coordinates of the point in the pixel coordinate system. R is the camera focal length, R is the rotation matrix, and T is the translation matrix. , , () are the coordinates of a point in the world coordinate system. and These are the x and y coordinates of the camera's focal length, respectively.

[0047] The present invention provides a method for generating underwater three-dimensional point clouds using a line laser scanning device. The method involves scanning a turbid underwater environment with a galvanometer laser while simultaneously capturing images with an underwater camera. The obtained image data is processed by the method to generate a three-dimensional point cloud map of the underwater scene surface.

[0048] Point cloud maps can be used by underwater fishing robots to estimate the location and size of seafood, and provide 3D coordinates to assist in the harvesting process.

[0049] A novel underwater image correction underwater camera calibration model is proposed. This model uses an approximate single-view camera model to correct water vapor refraction distortion and lens distortion. Combining an optical camera and structured light, it leverages the camera's optical capture capabilities and the relatively low impact of laser light on the underwater environment to generate a wide-area and high-precision 3D point cloud map. This model offers the following advantages:

[0050] 1. The parameters after camera calibration are reverse-mapped, and the pixel mean square error is only 0.4 pixels.

[0051] 2. It can generate point clouds with high robustness and generates point clouds quickly.

[0052] 3. The rotation of the galvanometer can be precisely controlled;

[0053] 4. The assembled chute and galvanometer support are very low cost through self-design and 3D printing. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor under the premise of the drawings.

[0055] Figure 1 is a structural diagram of a line laser scanning device;

[0056] Figure 2 is a line laser scanning device without a sealed transparent shell;

[0057] Figure 3 is an effect diagram of enhancing experimental data;

[0058] Figure 4 is a diagram of robust extraction by tracking laser trajectory;

[0059] Figure 5 is a diagram of center line extraction for the extracted laser strip;

[0060] Figure 6 is a close-up diagram of center line extraction for the extracted laser strip;

[0061] Figure 7 Line laser scanning imaging schematic diagram;

[0062] Figure 8 is a diagram of the process of underwater three-dimensional point cloud denoising filtering;

[0063] Figure 9 is an image before denoising (the red box is the target area);

[0064] Figure 10 is an image after denoising;

[0065] Figure 11 is a refraction model of air-waterproof shell-water multiple media;

[0066] Figure 12 is a shooting pose diagram of a calibration board;

[0067] Figure 13 is a calibration error analysis diagram;

[0068] Reference signs: 1, waterproof cabin shell, 2, camera module, 3, line laser, 4, bottom plate, 5, sliding groove, 6, galvanometer, 7, galvanometer support, 8, waterproof cabin cover. DETAILED DESCRIPTION

[0069] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0070] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. The description of the at least one exemplary embodiment is actually only illustrative, not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0071] Figure 1 is a structural diagram of a line laser scanning device;

[0072] Figure 2 is a line laser scanning device without a sealed transparent shell;

[0073] A line laser scanning device comprises:

[0074] A laser 3 for scanning a target;

[0075] A galvanometer 6 for refracting the laser emitted by the laser 3;

[0076] A controller for controlling the deflection amplitude and angular velocity of the galvanometer 6, and a support 7 arranged below the galvanometer 6;

[0077] A camera 2 for photographing the process of the laser line scanned by the galvanometer 6 traversing the target. A low-cost camera 2 is used, which facilitates the popularization and application of the project;

[0078] A sealed transparent shell 1 is arranged outside the line laser scanning device;

[0079] The sealed transparent shell 1 is arranged outside the point cloud generation module; the sealed transparent shell 1 can be applied in underwater scenes; the sealed transparent shell 1 can be made of glass; the sealed transparent shell 1 comprises a shell arranged around, and waterproof cabin covers 8 arranged above and below;

[0080] A bottom plate 4 is further arranged below the line laser scanning device;

[0081] Further, it also includes a galvanometer support structure for placing the galvanometer 6.

[0082] Further, it also includes a sliding module for the galvanometer support structure to slide. The sliding module is provided with a sliding groove on the cuboid structure; the galvanometer support structure is fixed on the sliding groove 5 by screws; the sliding groove 5 adopts a hollowed-out rounded rectangle.

[0083] The camera 2 is arranged above the laser 3.

[0084] The code aspects include galvanometer 6 control code, camera 2 calibration, laser line extraction, laser plane calibration, three-dimensional reconstruction, camera 2 distortion, double refraction, that is, outputting a digital signal through an arduino mainboard, and then converting the digital signal into a voltage signal through a digital-to-analog conversion module DAC, and then driving the galvanometer 6 to rotate through a driving board and a power board, and writing the required control code to control the rotation of the galvanometer 6.

[0085] A method for generating underwater three-dimensional point cloud of linear laser scanning device, comprising the following steps:

[0086] S1: calibrating the internal and external parameters of the camera under water based on the double refraction model of light under water, obtaining the distortion coefficient and internal and external parameters of the camera under water. The distortion coefficient can correct the internal and external parameters of the camera obtained by calibrating the geometric distortion caused by the camera lens, and the corresponding relationship between the image pixel coordinates and the three-dimensional coordinates in space can be established;

[0087] S2: processing the photographed image of the laser scanning target process, extracting the region of interest or features in the image, and performing target laser strip extraction;

[0088] The process of processing the photographed image of the laser scanning target process, extracting the region of interest or features in the image, and performing target laser strip extraction is as follows:

[0089] S21: preprocessing the photographed image of the laser scanning target process;

[0090] The preprocessing adopts an underwater light transmission model, and the specific expression is as follows:

[0091] (1)

[0092] Wherein: is the observed value of the pixel point coordinate in the underwater image;

[0093] is the true color of the point in the scene, is the transmission diagram, indicating the attenuation of light before reaching the camera;

[0094] is the attenuation coefficient of the water body, which is related to the wavelength; is the distance from the point to the camera; is the backscattered background light;

[0095] S22: image enhancement and laser reflection simulation, simulate the scattering and absorption process of laser line through water, recover the real laser energy, color compensation or waveband reconstruction of laser, to achieve the effect of enhancing experimental data, as shown in Figure 3 ;

[0096] S23: In the region of interest extraction stage, an adaptive brightness + color space fusion extraction algorithm is proposed, which uses morphological directional filter to exclude nonlinear interference, introduces spatial consistency discrimination, filters false lines using the geometric characteristics of laser lines, and uses inter-frame maximum value projection to highlight stable laser according to video acquisition characteristics, and tracks laser trajectory to achieve robust extraction; as shown in Figure 4 ;

[0097] S3: Laser strip center line extraction: center line extraction is performed on the extracted laser strip;

[0098] Figure 5 is the long-range view of the center line extraction of the extracted laser strip;

[0099] Figure 6 is the close-up view of the center line extraction of the extracted laser strip;

[0100] S4: Laser plane calibration;

[0101] S4: Laser plane calibration;

[0102] S5: Three-dimensional reconstruction:

[0103] After the parameters of the laser plane equation are determined in the initial state, the real-time parameters of each frame of the laser plane in the process of rotating scanning of the laser line are obtained according to the rotating axis of the laser plane and the rotating angular velocity of the galvanometer controlled by the mainboard, the two-dimensional points extracted from each frame of the laser line are combined with the internal and external parameters of the camera to establish the corresponding relationship between the image pixel coordinates and the three-dimensional coordinates in space, and the three-dimensional unit vector corresponding to the two-dimensional pixel coordinates is obtained. The three-dimensional unit vector and the laser plane equation are combined to obtain a unique three-dimensional point solution. The three-dimensional points of the whole line are generated in turn, and all the laser line scanning target images are processed to obtain the global three-dimensional point cloud, and the reconstruction of the three-dimensional point cloud is realized;

[0104] The expression of the laser space plane is as follows:

[0105] (2)

[0106] , The coordinate value of a certain point of the target in the camera coordinate system; A, B, C, and D are plane coefficients;

[0107] The parameter equation of the space straight line is as follows:

[0108] (3)

[0109] S is the conversion unit coefficient.

[0110] Figure 7 is a schematic diagram of line laser scanning imaging;

[0111] The application realizes the cooperation of cross-dimension information fusion and intelligent denoising by introducing two-dimensional image semantic information to assist three-dimensional point cloud noise removal.

[0112] S6: Underwater three-dimensional point cloud denoising filtering: filtering the global three-dimensional point cloud obtained to obtain the finally generated point cloud;

[0113] The process of denoising and filtering the global three-dimensional point cloud obtained to obtain the finally generated point cloud is as follows:

[0114] S61: Construct a high-quality point cloud mapping image, project the original three-dimensional point cloud to a two-dimensional plane, and realize the fusion of three-dimensional geometry and two-dimensional visual information;

[0115] S62: Use the SAM (Segment Anything Model) segmentation model to perform pixel-level segmentation on the mapping image to accurately locate the potential noise area;

[0116] ​S63: Based on the segmentation result, the region screening is performed, the suspected noise points are discriminated again in combination with the multi-scale context information and boundary consistency, and a noise region is obtained; the reliability of noise identification is improved;

[0117] S64: The noise region is mapped back to the three-dimensional space and removed, so as to realize efficient and high-precision point cloud denoising, realize global three-dimensional point cloud denoising filtering processing, and obtain the finally generated point cloud.

[0118] Figure 8 A process diagram for underwater three-dimensional point cloud denoising filtering;

[0119] Figure 9 An image before denoising (the red box is the target region);

[0120] Figure 10 A long-range image after denoising;

[0121] Further: the process of calibrating the internal and external parameters of the camera under water based on the double refraction model of light under water is as follows:

[0122] When working underwater, the instrument is sealed in a waterproof device, so that the underwater target and the laser scanning system are isolated in different refractive index media. When the camera shoots underwater objects, light passes through the horizontal plane glass waterproof cover and the glass waterproof cover-air interface. After being refracted twice, the object is finally imaged on the image surface of the camera. There is a refraction error when optical data is acquired, and it is of great significance to establish an effective refraction model for refraction compensation to improve the accuracy of the overall system; therefore, on the basis of considering the refractive index between multiple media and snell's law, a refraction model of air-waterproof shell-water multiple media is established, as shown in Figure 11 ;

[0123] S11: Calibrate the camera 2; acquire images of the calibration board at different angles, and calibrate the internal and external parameters of the camera 2 by using a camera 2 calibration method; the process of calibrating the internal and external parameters of the camera by using the camera calibration method is as follows:

[0124] As can be seen from Figure 3 , the imaging point of the underwater target Pw on the image plane is , if it is in the pinhole refraction model, the direct imaging point is . Underwater, after multiple refractions, the pinhole refraction model no longer holds. If no refraction compensation is performed, the obtained point is not the actual real point, i.e. the virtual point , and there is a relationship between

[0125]

[0126] in: θ is the vertical distance from the laser emission point to the imaging plane, and θ8 is the angle between the erroneous light transmission path corresponding to the virtual point and the horizontal direction;

[0127] Substitute the refraction compensation parameters into the camera calibration program, and then perform calibration on the dots on the dot calibration board taken from multiple angles, such as... Figure 12 The data is extracted, and after refraction compensation, the two-dimensional coordinates of the dots are obtained to get an array of real underwater dot pixel coordinates. This coordinate data is then used for calibration to obtain the camera's distortion coefficients and intrinsic and extrinsic parameters underwater. The obtained calibration results are quite accurate, with a mean square error of only 0.4 pixels for the remapping results. Figure 13 As shown;

[0128] S12: Underwater optical distortion correction: Correction of geometric distortion caused by the camera lens using distortion coefficients obtained through calibration. Furthermore, the camera calibration method of this application is simple and efficient, requiring no expensive precision calibration equipment, and is convenient and low-cost. By acquiring checkerboard images from different angles using multiple photographs, and optimizing intrinsic and extrinsic parameters using the least squares method, high calibration accuracy is achieved. It exhibits strong robustness to slight deformations and noise in the checkerboard pattern, and can overcome image noise and slight checkerboard deformation problems to a certain extent. It balances computational efficiency and accuracy.

[0129] The formula used to optimize the calibration of camera 2 and its intrinsic and extrinsic parameters using the least squares method is as follows:

[0130] (5)

[0131] in: (u, v) represents the value of the point along the Z-axis in the camera's 2-coordinate system, and (u, v) represents the pixel coordinates of the point on the image. and This indicates the pixel size of camera 2. :along The physical size corresponding to each pixel in the orientation. :along The physical size corresponding to each pixel in the direction. , () represents the coordinates of the point in the pixel coordinate system. It is the camera's focal length 2. R is the rotation matrix, T is the translation matrix, ( , , () are the coordinates of a point in the world coordinate system. = / , = / , and and are the focal lengths of the camera, usually in pixels. They are related to the physical focal length of the camera sensor (usually in millimeters) and the pixel size of the image (i.e., how many millimeters each pixel represents).

[0132] Further, the image is processed in grayscale, and the weighted center of the pixel grayscale in the target area of the captured laser line is calculated, which can achieve sub-pixel level accuracy. Compared with the simple pixel-based position determination, the accuracy is significantly improved, and it is suitable for scenes that require high positioning accuracy. To some extent, it has good robustness to changes in image lighting, noise, and local distortion, as the line laser center line extraction method uses the overall information of the region grayscale value, rather than the grayscale value of a single point or single line. The grayscale gravity method is simple, only needs to calculate the weighted average, and is suitable for real-time processing. Compared with complex feature extraction algorithms, it has faster calculation speed and is suitable for embedded systems or real-time applications. The calculation process is relatively simple and easy to implement. Without complex parameter adjustment or large amount of data training, it is very suitable for application in the lightweight environment of underwater robots.

[0133] Further, the process of extracting the center line of the target laser strip is as follows:

[0134] The image is enhanced and the laser reflection is simulated. The scattering and absorption process of the laser line through the water is simulated, the true laser energy is recovered in reverse, the laser is color compensated or band reconstructed, and the effect of enhancing the experimental data is achieved (such as Figure 3 ). In the interested region extraction stage, an adaptive brightness + color space fusion extraction algorithm is proposed, which uses morphological directional filter to exclude nonlinear interference, introduces spatial consistency discrimination, and filters false lines using the geometric characteristics of the laser line. According to the video acquisition characteristics, the maximum value projection between frames is used to highlight the stable laser, and the laser trajectory is tracked to realize robust extraction of the target laser strip region. The mean square gray gradient method is used to calculate the normal direction of the target laser strip at each point on the target laser strip skeleton, and the weighted gray gravity method is used along the normal direction of the target laser strip to obtain the initial point of the sub-pixel center of the target laser strip,

[0135] Finally, the initial point of the sub-pixel center of the target laser strip is segmented and fitted with a 3rd order polynomial curve, and the final sub-pixel center coordinates of the light strip are obtained, completing the center line extraction of the target laser strip.

[0136] The formula for extracting the center line of the laser strip is as follows:

[0137] (6)

[0138] Where: : is the laser strip center line point abscissa of the point : weight factor, indicating the weight of different points in the calculation. : point : coordinate value of the point, indicating the specific position of the point participating in the weighted average. : coordinate value of the point, indicating the specific position of the point participating in the weighted average.

[0139] : represents the number of rows of the image, usually along the vertical direction (Y-axis). In a two-dimensional image, "Y" is the horizontal line from top to bottom, i.e. the coordinates from the 1st row to the nth row. In image processing, "Y" usually represents the vertical coordinate of the image. : represents the number of columns of the image, usually along the horizontal direction (X-axis). In the image, "X" is the vertical line from left to right, i.e. the coordinates from the 1st column to the nth column.

[0140] : represents the number of columns of the image, usually along the horizontal direction (X-axis). In the image, "X" is the vertical line from left to right, i.e. the coordinates from the 1st column to the nth column. : represents the number of columns of the image, usually along the horizontal direction (X-axis). In the image, "X" is the vertical line from left to right, i.e. the coordinates from the 1st column to the nth column. : represents the number of columns of the image, usually along the horizontal direction (X-axis). In the image, "X" is the vertical line from left to right, i.e. the coordinates from the 1st column to the nth column. : represents the number of columns of the image, usually along the horizontal direction (X-axis). In the image, "X" is the vertical line from left to right, i.e. the coordinates from the 1st column to the nth column.

[0141] : represents the brightness value of the point (x, y) in the image. ∑: represents the sum of all points participating in the calculation. : represents the brightness value of the point (x, y) in the image. ∑: represents the sum of all points participating in the calculation.

[0142] Further: the formula used for underwater optical distortion correction is as follows:

[0143] (7)

[0144] The above formula (7) is transformed into:

[0145] (8)

[0146] This formula is a correction model for lens distortion (Distortion), especially the correction of radial distortion and tangential distortion in optical systems. Radial distortion (Radial Distortion): the center of the image is twisted outward or inward, commonly known as "barrel distortion" and "pincushion distortion". Tangential distortion (Tangential Distortion): due to the incomplete alignment of the lens and the sensor, resulting in the translation and tilt of the image pixels.

[0147] : represents the original image horizontal and vertical coordinates (uncorrected pixel coordinates). : represents the image coordinates after distortion correction (pixel coordinates after distortion elimination). : represents the image coordinates after distortion correction (pixel coordinates after distortion elimination). ​​​​, , : represent radial distortion coefficients. They describe the radial distortion (e.g. "barrel distortion" or "pincushion distortion") in the image caused by the lens; the radial distortion increases with the radius r from the center of the image; , : represent tangential distortion coefficients. They describe the tangential distortion caused by the imperfect alignment of the lens, i.e. some points in the image are shifted horizontally or vertically. r: the radius of the point to the center of the image, the calculation formula is: r^2 = x^2 + y^2.

[0148] Further: the formula of the fitted laser line plane is as follows:

[0149] (9)

[0150] The formula (7) is converted into the following formula:

[0151] (10)

[0152] Wherein: (x, y, z) is the coordinate value, , , ) are the parameters of the plane to be determined. S is the square of the value of all points to the fitted plane to be determined, the formula is to take the derivative of (x, y, z) respectively, and the minimum value of S corresponds to (x, y, z) is the most suitable laser plane coefficient, the left formula is to find the minimum value of S, and the right formula is obtained by taking the derivative, a total of n sets of three-dimensional points, I represents a particular set. , , , , , , The left formula is to find the minimum value of S, and the right formula is obtained by taking the derivative, a total of n sets of three-dimensional points, I represents a particular set.

[0153] ​​Considering the characteristics of underwater environment, the camera is always placed in a waterproof shell when pre-processing the image data. Therefore, the additional distortion caused by the refraction between water, waterproof shell and air is introduced into the system. When calibrating the underwater camera 2, the traditional calibration method regards the water-air refraction distortion as the lens distortion which introduces additional error. Therefore, we design a new underwater image correction underwater camera 2 calibration model (camera 2 calibration method + double refraction model), which uses the approximate single view camera 2 model to correct the water-air refraction distortion and lens distortion respectively. By calibrating the internal and external parameters of the camera 2, the refraction distortion correction is realized by establishing the underwater double refraction model. It is assumed that a light ray is emitted from a three-dimensional point and intersects the image plane at an image point. The distance between the waterproof glass and the camera 2 lens is represented by d. The thickness of the waterproof shell is much smaller than the depth of the world point. Therefore, the thickness of the waterproof shell can be assumed to be zero. Therefore, in the proposed underwater camera 2 calibration model, refraction should only occur once between water and air. In addition, it is assumed that the optical axis is perpendicular to the interface between water and air. According to Snell's law, the refraction that occurs between water and air can be quantitatively described. By virtually setting two virtual imaging planes, the refraction of light caused by the different refractive indices of water and air during camera 2 imaging is removed, and finally an approximate single view camera 2 imaging model is realized. Considering that the lens distortion and the distortion caused by the underwater-air refraction are divided into two parts of distortion to be corrected respectively, a higher correction accuracy is achieved.

[0154] Embodiment 1: First, take about 20 pictures of the round dot calibration board underwater to calibrate the camera 2 parameters.

[0155] Model the underwater camera 2 model of air-water-water shell refraction.

[0156] Use arduino_Nano to control the galvanometer laser line to scan the target scene, and the camera to capture and collect the scanning data. Fit the laser plane, calculate the parameters of each frame of laser plane through the rotation angular velocity, extract the unit vectors of three points on the laser line, and solve the coordinates of the three-dimensional points by combining the plane equations.

[0157] Iterate through the entire scene model in this way to generate a point cloud.

[0158] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for part or all of the technical features; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1.A method for generating an underwater three-dimensional point cloud of a line laser scanning device, characterized by: A linear laser scanning device, comprising: a laser for scanning a target; a galvanometer for refracting laser emitted by the laser; a controller for controlling the deflection amplitude and angular velocity of the galvanometer; a camera for capturing the process of the laser line refracted by the galvanometer scanning the target to obtain a captured image of the scanning target process; a sealed transparent housing is arranged outside the laser, the galvanometer, the controller and the camera; An underwater three-dimensional point cloud generation method, comprising the following steps: S1: calibrating the internal and external parameters of the camera under water to obtain the distortion coefficient and internal and external parameters of the camera under water, the distortion coefficient correcting the geometric distortion caused by the camera lens, the obtained internal and external parameters of the camera being calibrated, and the corresponding relationship between the image pixel coordinates and the three-dimensional coordinates in space being established; S2: processing the captured image of the laser scanning target process, extracting the region of interest or features in the image, and extracting the target laser strip; S3: extracting the center line of the extracted target laser strip; S4: capturing more than two images of the circular dot calibration board under different pose states, performing image preprocessing on the images, segmenting the laser strip, extracting the center line, obtaining a two-dimensional coordinate array, establishing the corresponding relationship between the image pixel coordinates and the three-dimensional coordinates in space according to the internal and external parameters obtained by camera calibration to obtain a three-dimensional coordinate array; using the three-dimensional coordinate array as input, fitting the plane equation of the laser line by least squares method to obtain the parameters of the plane equation; S5: after the parameters of the laser plane equation are determined in the initial state, the real-time parameters of each frame of the laser plane in the process of rotating scanning the target are obtained according to the rotation axis of the laser plane and the rotation angular velocity of the galvanometer controlled by the main board, the two-dimensional points after laser line extraction of each frame are combined with the internal and external parameters of the camera to establish the corresponding relationship between the image pixel coordinates and the three-dimensional coordinates in space, the three-dimensional unit vectors corresponding to the two-dimensional pixel coordinates are obtained, the three-dimensional unit vectors and the laser plane equation are combined to obtain a unique three-dimensional point solution, and the three-dimensional points of the whole line are generated in turn, and then all the laser line scanning target images are processed to obtain the global three-dimensional point cloud, and the three-dimensional point cloud is reconstructed; S6: performing denoising and filtering processing on the obtained global three-dimensional point cloud to obtain the finally generated point cloud. 2.The method of claim 1, wherein: The process of processing the captured image of the laser scanning target process, extracting the region of interest or features in the image, and extracting the target laser strip is as follows: S21: preprocessing the captured image of the laser scanning target process; The preprocessing adopts an underwater light transmission model, and the specific expression is as follows: wherein: are observed values of the pixel coordinates in the underwater image; a true color for the point in the scene, a transport map representing attenuation of light before reaching the camera; is the attenuation coefficient of the water body, which is a function of wavelength; is the distance of the point from the camera; is the backscattered background light; S22: enhancing and simulating laser reflection of the image, simulating the scattering and absorption process of the laser line passing through water, reversely restoring the real laser energy, color compensating or band reconstructing the laser to achieve the effect of enhancing the experimental data; S23: In the region of interest extraction stage, an adaptive brightness and color space fusion extraction algorithm is proposed, a morphological directional filter is used to exclude nonlinear interference, spatial consistency is introduced, the geometric characteristics of the laser line are used to filter false lines, and according to the video acquisition characteristics, the maximum value projection between frames is used to highlight the stable laser, and the laser trajectory is tracked to realize robust extraction. 3.The method of claim 1, wherein: The process of denoising and filtering the obtained global three-dimensional point cloud to obtain the finally generated point cloud is as follows: S61: Construct a high-quality point cloud mapping image, project the original three-dimensional point cloud to a two-dimensional plane, and realize three-dimensional geometry-two-dimensional vision information fusion; S62: The mapping image is segmented at the pixel level using a cut everything model to locate potential noise areas; S63: Based on the segmentation result, the region is screened, and the suspected noise points are discriminated again in combination with multi-scale context information and boundary consistency to obtain a noise region; S64: The noise region is mapped back to the three-dimensional space and removed to realize denoising and filtering of the global three-dimensional point cloud, and obtain the finally generated point cloud. 4.The method of claim 1, wherein: The line laser scanning device also includes a galvanometer support structure for placing the galvanometer, and a sliding module for sliding the galvanometer support structure. 5.The method of claim 1, wherein: The process of calibrating the internal and external parameters of the camera underwater to obtain the distortion coefficient and internal and external parameters of the camera underwater is as follows: The imaging point of the underwater target Pw on the image plane is , The horizontal and vertical coordinates of the pixel point are respectively x and y; if the imaging point is in the pinhole refraction model, the direct imaging point is In water, after multiple refraction, the pinhole refraction model no longer holds; if no refraction compensation processing is performed, the obtained point is not the actual real point, i.e., a virtual point and There is a relationship between the virtual point and the real point. wherein: is the vertical distance from the laser emission point to the imaging plane, and θ8is the angle between the erroneous light transmission route corresponding to the virtual point and the horizontal direction. Substitute the parameters of the refraction compensation into the camera calibration program, extract the circle points in the circle point calibration plate, obtain the real circle point pixel coordinate array after refraction compensation of the two-dimensional coordinates of the circle points, and then use these coordinate data for calibration to obtain the distortion coefficient and internal and external parameters of the camera underwater; Underwater optical distortion correction: correct the geometric distortion caused by the camera lens by using the distortion coefficient obtained by calibration; Refraction compensation: establish an underwater double refraction model, use the model to calculate the real direction of light underwater from the image coordinates captured by the camera, and remap it to the corrected real image coordinates. 6.The method of claim 1, wherein: The process of center line extraction of the extracted target laser strip is as follows: The mean square gray gradient method is used to calculate the normal direction of the target laser strip at each point in the target laser strip region, and the weighted gray gravity center method is used along the normal direction of the target laser strip to obtain the sub-pixel center initial point of the target laser strip, Finally, the sub-pixel center initial point of the target laser strip is segmented and fitted by a 3rd order polynomial curve to obtain the sub-pixel center coordinates of the final light strip, and the center line extraction of the target laser strip is completed. 7.The method of claim 1, wherein: The formula used for calibrating the internal and external parameters of the camera underwater is as follows: wherein: is the value of the point on the target object in the Z-axis direction in the camera coordinate system, , ) is the pixel coordinate of the point on the image, and denote the pixel size of the camera, : is the physical size corresponding to each pixel along the direction; : is the physical size corresponding to each pixel along the direction, , ) is the coordinate of the circle point in the pixel coordinate system, is the focal length of the camera, R is the rotation matrix, T is the displacement matrix, , , ) is the coordinate of the point in the world coordinate system, and are the horizontal and vertical coordinates of the focal length of the camera, respectively.

Images