A deep-sea three-dimensional laser scanning imaging method

Abstract

This invention discloses a deep-sea 3D laser scanning imaging method. The system includes a laser, a camera, an altimeter, and a gyroscope, all operating underwater. The laser generates a straight line of laser light; the camera captures the laser image produced by the laser, and the laser and camera are fixed at a preset angle by a fixing device and remain unchanged; the altimeter measures the system's altitude; and the gyroscope measures the inclination angle between the system and the horizontal plane caused by seawater fluctuations. The method includes the following steps: S1, the camera captures the laser image produced by the laser, and the altimeter and gyroscope respectively measure the system's altitude and inclination angle; S2, correcting the image captured by the camera; and S3, constructing a 3D model based on the corrected image. The deep-sea image correction algorithm provided by this invention can reduce labor costs while improving recognition accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a deep-sea three-dimensional laser scanning imaging method. Background Art

[0002] The ocean is rich in natural resources, but because they are much more difficult to exploit than land resources, many resources remain untapped or even undiscovered. To exploit these resources, the development of underwater imaging technology is crucial. Researchers use a variety of sensors to survey the seabed and identify the resources we need.

[0003] Computer vision is widely used in the air, but its underwater application presents numerous challenges. On the one hand, images captured underwater are blurred due to water absorption and scattering. On the other hand, for economic and installation convenience, cameras are typically mounted in waterproof enclosures with flat, sealed glass. Therefore, light from an object passes through water, glass, and air to reach the camera's imaging plane. Light refracts at the water-glass and glass-air interfaces, causing the light from the object to no longer converge at a single projection center. Consequently, image formation violates the single viewpoint (SVP) assumption of the pinhole camera model, leading to significant distortion of underwater images. Consequently, a camera calibrated in air loses its parameters upon entering water. Although using a perspective camera model for underwater scenes introduces systematic model errors, numerous works exist in the literature where methods designed for perspective cameras have been applied to underwater images. Distortion of refracted images is often compensated by adjusting the camera's intrinsic parameters.

[0004] Unlike land, the deep ocean is not only dangerous but also characterized by extremely low light levels, making conventional imaging technology difficult to apply. Furthermore, powering submarine detectors is challenging, requiring numerous devices, all of which must minimize power consumption to maximize detection time. Summary of the Invention

[0005] In view of this, the present invention proposes a deep-sea 3D laser scanning imaging method. The deep-sea 3D laser scanning imaging system includes a laser, a camera, an altimeter, and a gyroscope, all of which operate underwater. The laser generates a straight line laser; the camera captures the laser image produced by the laser. The laser and the camera are fixed at a preset angle by a fixing device and remain unchanged; the altimeter measures the height of the system; and the gyroscope measures the inclination angle between the system and the horizontal plane caused by seawater fluctuations. The method includes the following steps:

[0006] S1, the camera captures the laser image emitted by the laser, and the altimeter and gyroscope measure the system height and inclination respectively;

[0007] S2, correcting the image captured by the camera;

[0008] S3, constructs a three-dimensional model based on the corrected image.

[0009] Preferably, the laser, camera, altimeter and gyroscope have the same frame rate. When the frame rates are different, a synchronized frame rate is obtained by interpolation.

[0010] Preferably, when the laser in S1 irradiates objects at different heights, the pattern of the laser line presented on the camera screen will change, and the relationship between the height and the position of the laser line in the image is:

[0011] Where Δh is the object height, h is the maximum height that the system can scan, Δd is the distance the laser line deviates from the center of the image, and d is the distance between the camera and the laser center.

[0012] Preferably, said S2 comprises the following steps:

[0013] S21, binary separation of the foreground laser line and the environmental background, and selection of the ROI region containing the laser line;

[0014] S22, correcting the image according to the data of the altimeter and the gyroscope.

[0015] Preferably, the S21 includes converting the image from an RGB image to a grayscale image. The image captured by the camera contains three channels, red R, green G and blue B, that is, an RGB image. The formula for converting the RGB image to a grayscale image is:

[0016] Gray=R*0.299+G*0.587+B*0.114

[0017] In order to reduce floating-point operations and increase computing speed, the equivalent formula is used:

[0018] Gray=(R*30+G*59+B*11+50) / 100

[0019] Among them, Gray represents the required grayscale image, R, G, and B represent the three channels of the RGB color image respectively;

[0020] Then perform binarization on the grayscale image, including setting the brightness threshold to find the position of the laser line. The binarization formula is:

[0021]

[0022] Where p is the value of a single pixel in the grayscale image, and thres is the set brightness threshold.

[0023] Preferably, in S22, the image is scaled according to the altimeter data to achieve altitude correction; and the image is affine transformed according to the gyroscope data to achieve tilt correction.

[0024] Preferably, the height correction is specifically as follows: when the height of the camera from the scanning area is lower than a preset height, the image of the captured object will be enlarged, the image will be reduced according to the ratio of the actual height to the preset height, and then zeros will be added to the surrounding positions of the image;

[0025] When the height of the camera from the scanning area is higher than the preset height, the image of the object captured will become smaller, and the image will be enlarged according to the ratio of the actual height to the preset height.

[0026] Preferably, the tilt correction is specifically as follows: the affine transformation formula is:

[0027]

[0028] Among them, (x, y) is the pixel coordinate before correction, (x′, y′) is the pixel coordinate after correction, and M is the affine matrix.

[0029] Preferably, the step S3 includes calibrating four parameters by simulating underwater calibration on land. The four parameters include:

[0030] The first two calibration parameters are the camera's intrinsic and extrinsic parameters. The camera's intrinsic parameter is the correspondence between real-world coordinates and image coordinates; the camera's extrinsic parameter is the conversion relationship between image coordinates and the established 3D model coordinates. The third parameter is the conversion relationship between the light plane and the 3D model coordinates. The light plane is the plane formed by the laser's coordinates and the laser line. Through this plane, the height information of the object illuminated by the laser line can be calculated. The fourth parameter is the movement direction and speed of the entire system, which is also the speed of the laser scanning.

[0031] The above four parameters can be used to calculate the three-dimensional model coordinates corresponding to the laser line irradiation position based on the captured laser line image, and obtain the X coordinate graph, Y coordinate graph and Z coordinate graph;

[0032] According to the obtained X, Y, and Z coordinates, all coordinate points can be fused to generate a three-dimensional model.

[0033] Preferably, the step S3 includes the following steps:

[0034] S31, obtaining a depth parameter according to the corrected laser line image and calibration data;

[0035] S32, mapping the two-dimensional image data to a three-dimensional space according to the depth parameter to obtain the three-dimensional coordinates of all points;

[0036] S33, generating a three-dimensional model according to the obtained three-dimensional coordinates.

[0037] Beneficial effect: The above-mentioned deep-sea three-dimensional laser scanning imaging method can effectively solve the modeling problems in deep-sea environments. When performing laser scanning modeling on land, the camera and the laser are stably fixed together, with only very slight movement. However, the underwater environment is not stable, and the equipment will pitch and roll with the movement of the sea water underwater, which will affect the results of the three-dimensional modeling and cause a large error with the actual environment. In order to solve this problem, the present invention designs a solution to correct the three-dimensional modeling results through the data of the altimeter and gyroscope. While the laser and camera are scanning and shooting, they read the data of the altimeter and gyroscope in real time, and calculate the deviation of the scanning device itself relative to the parallel position based on the data of these two sensors. Based on this deviation, the parameters in the three-dimensional modeling are modified to compensate for the error caused by the jitter of the sea water. The power of the laser itself is not high, and after correction, it can achieve very high accuracy at the same time, and the system as a whole is a low-power and high-precision underwater imaging system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Schematic diagram of laser scanning of a deep-sea 3D laser scanning imaging method according to an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of the positions of a camera and a laser in a deep-sea 3D laser scanning imaging method according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the laser line captured by the camera of the deep-sea three-dimensional laser scanning imaging method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] On the contrary, the present invention covers any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention as defined by the claims. Furthermore, to facilitate a better understanding of the present invention, certain specific details are described in detail below in the detailed description of the present invention. Those skilled in the art will be able to fully understand the present invention without these details.

[0043] To achieve the above object, the technical solution of the present invention is as follows:

[0044] A deep-sea 3D laser scanning imaging method is based on a deep-sea 3D laser scanning imaging system comprising a laser 2, a camera 1, an altimeter, and a gyroscope, all of which operate underwater. The laser 2 generates a straight line laser; the camera 1 captures the laser image produced by the laser 2. The laser 2 and the camera 1 are fixed at a preset angle by a fixing device and remain unchanged. Figure 1 、 Figure 2 , α is the angle between the laser 2 and the camera 1; the altimeter measures the height of the system; the gyroscope measures the inclination angle between the system and the horizontal plane caused by seawater fluctuations; the method comprises the following steps:

[0045] S1, the camera captures the laser image emitted by the laser, and the altimeter and gyroscope measure the system height and inclination respectively;

[0046] S2, correcting the image captured by the camera;

[0047] S3, constructs a three-dimensional model based on the corrected image.

[0048] The laser, camera, altimeter, and gyroscope have the same frame rate. When the frame rates are different, the synchronized frame rate is obtained by interpolation.

[0049] When the laser hits objects at different heights, the camera will capture different patterns. Based on the principle of camera imaging, the height of the object can be calculated, thus obtaining the three-dimensional z coordinates of different objects.

[0050] The x- and y-coordinates of the object are obtained from the two-dimensional coordinates of the image itself according to the calibration parameters of the camera, and an underwater three-dimensional model can be constructed based on these coordinates.

[0051] The underwater environment is unstable, and the device will pitch and roll with the movement of the water. This can affect the 3D modeling results, resulting in significant errors compared to the actual environment. To address this issue, the present invention designs a solution that uses data from an altimeter and gyroscope to correct the 3D modeling results.

[0052] While the laser and camera are scanning and shooting, they read the data from the altimeter and gyroscope in real time. Based on the data from these two sensors, they calculate the deviation of the scanning device itself from the parallel position. Based on this deviation, they modify the parameters during 3D modeling to compensate for the error caused by the jitter of the sea water.

[0053] The correction based on the altimeter is mainly to correct the z coordinate. When the entire system is lower than the target altitude due to seawater fluctuations, the z coordinate increases by a corresponding value. When the entire system is higher than the target altitude due to seawater fluctuations, the z coordinate decreases by a corresponding value.

[0054] The correction of the z coordinate is reflected in the image, which is essentially scaling the image. When the distance is far, the captured image is smaller, and when the distance is large, the captured image is larger. According to the principle of triangular similarity, the image can be scaled proportionally.

[0055] Correcting the pitch error of an image mainly involves correcting the x-coordinate and y-coordinate of the image, and transforming the coordinates containing the pitch error back to normal coordinates.

[0056] The image processing technology used is mainly affine transformation. The world coordinates corresponding to the position where the laser line is projected are calculated through the current pitch angle, and then the coordinates of the three points are obtained to obtain the affine matrix, and then the image is corrected to the correct position.

[0057] In a specific embodiment, when the laser in S1 shines on objects at different heights, the pattern of the laser line displayed on the camera screen will change, see Figure 3 , where P is a schematic diagram of the laser line irradiating the object, and P′ is a schematic diagram of the laser line captured by the camera.

[0058] The relationship between the height and the position of the laser line in the image Image is:

[0059]

[0060] Where Δh is the height of the object, h is the maximum height that the system can scan, Δd is the distance the laser line deviates from the center of the image, and d is the center distance between camera 1 and laser 2.

[0061] S2 includes the following steps:

[0062] S21, binary separation of the foreground laser line and the environmental background, and selection of the ROI region containing the laser line;

[0063] S22, correcting the image according to the data of the altimeter and the gyroscope.

[0064] S21 includes converting the image from RGB image to grayscale image. The image captured by the camera contains three channels, red R, green G and blue B, that is, RGB image. The formula for converting RGB image to grayscale image is:

[0065] Gray=R*0.299+G*0.587+B*0.114

[0066] In order to reduce floating-point operations and increase computing speed, the equivalent formula is used:

[0067] Gray=(R*30+G*59+B*11+50) / 100

[0068] Among them, Gray represents the required grayscale image, R, G, and B represent the three channels of the RGB color image respectively;

[0069] Then perform binarization on the grayscale image, including setting the brightness threshold to find the position of the laser line. The binarization formula is:

[0070]

[0071] Where p is the value of a single pixel in the grayscale image, and thres is the set brightness threshold.

[0072] In S22 , the image is scaled according to the altimeter data to achieve altitude correction; and the image is affine transformed according to the gyroscope data to achieve inclination correction.

[0073] The height correction is as follows: when the height of the camera from the scanning area is lower than the preset height, the image of the object captured will become larger. The image is reduced according to the ratio of the actual height to the preset height, and then zeros are added to the surrounding positions of the image.

[0074] When the height of the camera from the scanning area is higher than the preset height, the image of the object captured will become smaller, and the image will be enlarged according to the ratio of the actual height to the preset height.

[0075] The specific inclination correction is: the affine transformation formula is:

[0076]

[0077] Wherein, (x, y) is the pixel coordinate before correction, (x′, y′) is the pixel coordinate after correction, and M is the affine matrix.

[0078] S3 includes calibration of four parameters, which are calibrated by simulating underwater on land. The four parameters include:

[0079] The first two calibration parameters are the camera's intrinsic and extrinsic parameters. The camera's intrinsic parameter is the correspondence between real-world coordinates and image coordinates; the camera's extrinsic parameter is the conversion relationship between image coordinates and the established 3D model coordinates. The third parameter is the conversion relationship between the light plane and the 3D model coordinates. The light plane is the plane formed by the laser's coordinates and the laser line. Through this plane, the height information of the object illuminated by the laser line can be calculated. The fourth parameter is the movement direction and speed of the entire system, which is also the speed of the laser scanning.

[0080] The above four parameters can be used to calculate the three-dimensional model coordinates corresponding to the laser line irradiation position based on the captured laser line image, and obtain the X coordinate graph, Y coordinate graph and Z coordinate graph;

[0081] According to the obtained X, Y, and Z coordinates, all coordinate points can be fused to generate a three-dimensional model.

[0082] Specific calibration includes:

[0083] (1) Camera calibration

[0084] Place the camera in a pool of water at least 5 meters long, then place the calibration plate 5 meters away. Take photos of the calibration plate at different positions, ensuring that the calibration plate covers the entire camera's field of view as much as possible.

[0085] (2) Light plane calibration

[0086] Take photos of the laser line at a distance of 3m and 4m respectively, and take photos of the calibration plate at a distance of 3m and 4m at the same time.

[0087] (3) Calibration of system movement posture

[0088] Place the calibration plate at a certain position, take a photo, then move the entire system a certain step length and take another image. The system's movement posture can be calibrated based on the two images.

[0089] S3 includes the following steps:

[0090] S31, obtaining a depth parameter according to the corrected laser line image and calibration data;

[0091] S32, mapping the two-dimensional image data to a three-dimensional space according to the depth parameter to obtain the three-dimensional coordinates of all points;

[0092] S33, generating a three-dimensional model according to the obtained three-dimensional coordinates.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A deep-sea three-dimensional laser scanning imaging method, characterized in that: The deep-sea 3D laser scanning imaging system includes a laser, a camera, an altimeter, and a gyroscope, all of which operate underwater. The laser generates a straight line of laser light; the camera captures the laser image produced by the laser; the laser and the camera are fixed at a preset angle by a fixing device and remain unchanged; the altimeter measures the height of the system; and the gyroscope measures the inclination angle between the system and the horizontal plane caused by seawater fluctuations. The method includes the following steps: S1, the camera captures the laser image emitted by the laser, and the altimeter and gyroscope measure the system height and inclination respectively; S2, correcting the image captured by the camera; S3, constructing a three-dimensional model based on the corrected image; The S2 comprises the following steps: S21, binary separation of the foreground laser line and the background, and selection of the ROI region containing the laser line; S22, correcting the image based on the data from the altimeter and gyroscope; In the step S22, the image is scaled according to the altimeter data to achieve altitude correction; and the image is affine transformed according to the gyroscope data to achieve inclination correction; The height correction is as follows: when the height of the camera from the scanning area is lower than the preset height, the image of the object captured will be enlarged, the image will be reduced according to the ratio of the actual height to the preset height, and then zeros will be added to the surrounding positions of the image; When the height of the camera from the scanning area is higher than the preset height, the image of the object captured will become smaller, and the image will be enlarged according to the ratio of the actual height to the preset height.

2. The deep-sea three-dimensional laser scanning imaging method according to claim 1, characterized in that: The laser, camera, altimeter and gyroscope have the same frame rate. When the frame rates are different, a synchronized frame rate is obtained by interpolation.

3. The deep-sea three-dimensional laser scanning imaging method according to claim 1, characterized in that: When the laser in S1 is irradiated on objects at different heights, the pattern of the laser line on the camera screen will change. The relationship between the height and the position of the laser line in the image is: ; in, is the object height, is the maximum height that the system can scan, is the distance the laser line deviates from the center of the image, is the center distance between the camera and the laser.

4. The deep-sea three-dimensional laser scanning imaging method according to claim 1, characterized in that: The S21 includes converting the image from an RGB image to a grayscale image. The image captured by the camera contains three channels, red R, green G and blue B, that is, an RGB image. The formula for converting the RGB image to a grayscale image is: ； In order to reduce floating-point operations and increase computing speed, the equivalent formula is used: ； Among them, Gray represents the required grayscale image, and R, G, and B represent the three channels of the RGB image respectively; Then perform binarization on the grayscale image, including setting the brightness threshold to find the position of the laser line. The binarization formula is: ； in, is the value of a single pixel in the grayscale image, thres The brightness threshold to set.

5. The deep-sea three-dimensional laser scanning imaging method according to claim 1, characterized in that: The tilt correction is specifically as follows: the affine transformation formula is: ； in, is the pixel coordinate before correction, is the corrected pixel coordinate, is an affine matrix.

6. The deep-sea three-dimensional laser scanning imaging method according to claim 1, characterized in that: The S3 step includes calibrating four parameters by simulating underwater calibration on land. The four parameters include: The first two calibration parameters are the camera's intrinsic and extrinsic parameters. The camera's intrinsic parameter is the correspondence between real-world coordinates and image coordinates; the camera's extrinsic parameter is the conversion relationship between image coordinates and the established 3D model coordinates. The third parameter is the conversion relationship between the light plane and the 3D model coordinates. The light plane is the plane formed by the laser's coordinates and the laser line. Through this plane, the height information of the object illuminated by the laser line can be calculated. The fourth parameter is the movement direction and speed of the entire system, which is also the speed of the laser scanning. The above four parameters can be used to calculate the three-dimensional model coordinates corresponding to the laser line irradiation position based on the captured laser line image, and obtain the X coordinate, Y coordinate and Z coordinate; According to the obtained X, Y, and Z coordinates, all coordinate points can be fused to generate a three-dimensional model.

7. The deep-sea three-dimensional laser scanning imaging method according to claim 1, characterized in that: The S3 includes the following steps: S31, obtaining a depth parameter according to the corrected laser line image and calibration data; S32, mapping the two-dimensional image data to a three-dimensional space according to the depth parameter to obtain the three-dimensional coordinates of all points; S33, generating a three-dimensional model according to the obtained three-dimensional coordinates.

Images