Underwater environment three-dimensional reconstruction method, device and equipment and storage medium

Abstract

The invention discloses an underwater environment three-dimensional reconstruction method, device and equipment and a storage medium, and relates to the technical field of underwater robots, and the underwater environment three-dimensional reconstruction method comprises the steps: transmitting laser to a target object, and collecting a laser reflection image; extracting pixel coordinates of laser stripes from the laser reflection image; calculating a three-dimensional space coordinate of the laser point based on the pixel coordinate and a camera internal reference matrix; generating a control instruction according to the depth information of the three-dimensional space coordinates, and sending the control instruction to an underwater vehicle to enable the underwater vehicle to execute equidistant diameter circular motion and collect and feed back three-dimensional point cloud data under different poses; and fusing the three-dimensional point cloud data under different poses to obtain a target three-dimensional point cloud model, and completing three-dimensional reconstruction. According to the invention, three-dimensional reconstruction of the underwater environment can be efficiently and accurately realized at low cost, and powerful technical support is provided for underwater exploration and scientific research.

Description

Technical Field

[0001] The present application relates to the technical field of underwater robots, and particularly to a three-dimensional reconstruction method, device, equipment and storage medium for underwater environment. Background Art

[0002] The complexity of the underwater environment has brought many limitations to scientific research such as underwater resource development and geological surveys. During underwater operations, people often use underwater robots equipped with various sensors to obtain multi-source information to reduce the impact of complex underwater scenes on operations. As a technology capable of constructing a three-dimensional stereo scene around a target object, three-dimensional reconstruction technology can non-contactedly obtain surrounding environment information and establish a three-dimensional map, and is widely used in fields such as submarine pipeline maintenance and dam crack detection.

[0003] Currently, underwater three-dimensional scene reconstruction technologies are mainly divided into two categories: active and passive. Active three-dimensional reconstruction emits light waves to a target object through a sensor and receives the returned light waves to obtain the depth information of the object. Passive three-dimensional reconstruction uses devices such as cameras to obtain video image sequences and recovers three-dimensional scene information from two-dimensional images.

[0004] Although the existing underwater three-dimensional reconstruction technologies have made certain progress, there are still some problems. Due to the characteristics of turbidity and refraction in the underwater environment, conventional vision technologies are difficult to accurately obtain the depth information of objects underwater. In addition, the existing active and passive three-dimensional reconstruction technologies have deficiencies in terms of accuracy, operation complexity, and equipment cost. For example, although sonar devices can penetrate turbid water bodies, their resolution is low and it is difficult to provide high-precision three-dimensional information; while devices such as structured light have high accuracy, but are costly and are easily interfered with in the underwater environment. Therefore, how to efficiently, accurately and low-costly achieve three-dimensional reconstruction of the underwater environment has become an urgent problem to be solved.

[0005] The above content is only used to assist in understanding the technical solution of the present application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The purpose of the present application is to provide a three-dimensional reconstruction method, device, equipment and storage medium for underwater environment, aiming to solve the technical problem of how to efficiently, accurately and low-costly achieve three-dimensional reconstruction of the underwater environment.

[0007] To achieve the above purpose, the present application proposes a three-dimensional reconstruction method for underwater environment, the method comprising:

[0008] Emitting a laser to a target object and collecting a laser reflection image;

[0009] Extracting the pixel coordinates of the laser stripe from the laser reflection image;

[0010] Based on the pixel coordinates and the camera intrinsic matrix, calculate the three-dimensional spatial coordinates of the laser point;

[0011] Generate a control command according to the depth information of the three-dimensional spatial coordinates, and send the control command to the underwater vehicle so that the underwater vehicle performs an equidistant diameter circular motion, collects three-dimensional point cloud data in different poses, and feeds back;

[0012] Fuse the three-dimensional point cloud data in different poses to obtain a target three-dimensional point cloud model, and complete three-dimensional reconstruction.

[0013] In one embodiment, the step of calculating the three-dimensional spatial coordinates of the laser point based on the pixel coordinates and the camera intrinsic matrix includes: obtaining the camera intrinsic matrix according to the Zhang Zhengyou calibration method; performing oblique laser triangulation ranging on the pixel coordinates based on the camera intrinsic matrix to obtain the depth information corresponding to the pixel coordinates; constructing a three-dimensional point correspondence relationship between the pixel coordinate system and the camera coordinate system according to the camera intrinsic matrix and the depth information; calculating the three-dimensional spatial coordinates of the laser point according to the three-dimensional point correspondence relationship and the pixel coordinates.

[0014] In one embodiment, the step of performing oblique laser triangulation ranging on the pixel coordinates based on the camera intrinsic matrix to obtain the depth information corresponding to the pixel coordinates includes: determining the angle between the laser emission direction and the camera optical axis according to the relative position parameters of the camera and the laser; based on the camera intrinsic matrix, inverse-project the pixel coordinates to the camera coordinate system to obtain the imaging light path direction vector; construct a laser light path direction vector according to the angle, and form a triangulation ranging geometric model with the imaging light path direction vector; obtain the depth information corresponding to the pixel coordinates according to the triangulation ranging geometric model.

[0015] In one embodiment, the step of constructing a laser light path direction vector according to the angle and forming a triangulation ranging geometric model with the imaging light path direction vector includes: constructing a laser light path direction vector according to the angle; establishing an intersection constraint condition of the laser light path direction vector and the imaging light path direction vector in three-dimensional space; generating a triangulation ranging geometric model of the laser light path and the camera imaging light path according to the laser light path direction vector, the imaging light path direction vector, and the intersection constraint condition.

[0016] In one embodiment, the step of extracting the pixel coordinates of the laser stripe from the laser reflection image includes: performing grayscale processing on the laser reflection image to obtain a grayscale distribution image; performing binary segmentation on the grayscale distribution image based on a preset laser spot intensity threshold to obtain a laser stripe mask region; scanning the laser stripe mask region row by row, and extracting the continuous pixel region with the largest grayscale value in each row as a candidate stripe segment; fitting the center line of the candidate stripe segment to obtain the center line coordinates of the laser stripe in the image; and extracting the pixel coordinates of the laser stripe at a preset sampling interval based on the center line coordinates.

[0017] In one embodiment, the step of generating a control instruction according to the depth information of the three-dimensional space coordinates, sending the control instruction to an underwater vehicle to make the underwater vehicle perform an equidistant diameter circular motion, collecting three-dimensional point cloud data at different poses and feeding back includes: determining the real-time distance between the target object and the underwater vehicle according to the depth information of the three-dimensional space coordinates; in the case where the real-time distance deviates from a preset distance threshold, determining a speed adjustment parameter and a direction adjustment parameter according to the real-time distance and the preset distance threshold; generating a control instruction according to the speed adjustment parameter and the direction adjustment parameter; and sending the control instruction to the underwater vehicle to make the underwater vehicle perform an equidistant diameter circular motion, collect three-dimensional point cloud data at different poses and feed back.

[0018] In one embodiment, the step of fusing the three-dimensional point cloud data at different poses to obtain a target three-dimensional point cloud model and complete three-dimensional reconstruction includes: obtaining the rotation matrix and translation vector of the underwater vehicle at different poses; according to the rotation matrix and the translation vector, converting each frame of the three-dimensional point cloud data at different poses to the coordinate system of the first frame to obtain reference point cloud data; performing outlier filtering and multi-view registration on the reference point cloud data to obtain target point cloud data; and superimposing and fusing the target point cloud data to obtain a target three-dimensional point cloud model and complete three-dimensional reconstruction.

[0019] In addition, to achieve the above object, the present application further provides a three-dimensional reconstruction device for an underwater environment. The device includes: a laser emission module for emitting laser to a target object and collecting a laser reflection image; a coordinate extraction module for extracting pixel coordinates of a laser stripe from the laser reflection image; a coordinate calculation module for calculating three-dimensional spatial coordinates of a laser point based on the pixel coordinates and an internal camera parameter matrix; a collaborative control module for generating a control instruction according to depth information of the three-dimensional spatial coordinates, and sending the control instruction to an underwater vehicle to make the underwater vehicle perform an equidistant diameter circular motion, collect three-dimensional point cloud data in different poses, and give feedback; and a fusion modeling module for fusing the three-dimensional point cloud data in different poses to obtain a target three-dimensional point cloud model, thereby completing three-dimensional reconstruction.

[0020] In addition, to achieve the above object, the present application further provides a three-dimensional reconstruction device for an underwater environment. The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. The computer program is configured to implement the steps of the three-dimensional reconstruction method for an underwater environment as described above.

[0021] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the three-dimensional reconstruction method for an underwater environment as described above are implemented.

[0022] In addition, to achieve the above object, the present application further provides a computer program product. The computer program product includes a computer program. When the computer program is executed by a processor, the steps of the three-dimensional reconstruction method for an underwater environment as described above are implemented.

[0023] One or more technical solutions proposed by the present application have at least the following technical effects:

[0024] First, the monocular line laser device emits laser light towards the target object and captures the laser reflection image. In this process, by emitting laser pulses and capturing the reflected light, high-contrast feature information on the surface of the target object is obtained, providing the basic data for subsequent 3D reconstruction. Next, the pixel coordinates of the laser stripe are extracted from the captured laser reflection image, and the position of the laser stripe is extracted through image processing algorithms, providing accurate input data for subsequent depth calculation. Then, based on the extracted pixel coordinates and the camera internal parameter matrix, the three-dimensional spatial coordinates of the laser points are calculated, mapping the feature points in the two-dimensional image into the three-dimensional space. After that, control instructions are generated according to the depth information of the laser points and these instructions are sent to the underwater vehicle to make it perform an equidistant diameter circular motion. The three-dimensional point cloud data is collected and fed back at different poses. By dynamically adjusting the motion trajectory of the underwater vehicle, high-quality three-dimensional point cloud data is uniformly collected at different angles and positions, reducing the data acquisition error caused by distance changes. Finally, the three-dimensional point cloud data at different poses is fused to generate the target three-dimensional point cloud model, thus completing the 3D reconstruction. This process not only improves the integrity and accuracy of the 3D reconstruction, but also provides high-quality three-dimensional data support for subsequent analysis and applications. This application can efficiently, accurately and low-costly achieve 3D reconstruction of the underwater environment, providing strong technical support for underwater exploration and scientific research. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0027] Figure 1 It is a schematic flowchart provided for Embodiment 1 of the method for 3D reconstruction of the underwater environment in this application;

[0028] Figure 2 It is a schematic diagram of the dynamic adjustment of the AUV based on depth information provided for Embodiment 1 of the method for 3D reconstruction of the underwater environment in this application;

[0029] Figure 3 It is a schematic flowchart provided for Embodiment 2 of the method for 3D reconstruction of the underwater environment in this application;

[0030] Figure 4 It is a schematic flowchart of the coordinate transformation provided for Embodiment 2 of the method for 3D reconstruction of the underwater environment in this application;

[0031] Figure 5 Schematic diagram of the brief process of the three-dimensional reconstruction method for the underwater environment provided in the second embodiment of the present application;

[0032] Figure 6 Schematic diagram of the module structure of the three-dimensional reconstruction device for the underwater environment in the embodiment of the present application;

[0033] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the three-dimensional reconstruction method for the underwater environment in the embodiment of the present application.

[0034] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Specific embodiments

[0035] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0036] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the drawings of the specification and specific embodiments.

[0037] The complexity of the underwater environment requires the use of underwater robots equipped with a variety of sensors for operation. Three-dimensional reconstruction technology obtains environmental information through non-contact methods and is widely used in seabed maintenance and detection. Existing underwater three-dimensional reconstruction technologies are divided into two types: active and passive. The active type uses sensors to emit and receive light waves to obtain depth information, while the passive type restores the three-dimensional scene from the image sequence obtained by the camera. However, these technologies face challenges when applied underwater, including inaccurate depth information acquisition due to turbid water quality and refraction, low resolution of sonar devices, and high costs and susceptibility to interference of high-precision devices such as structured light. These problems limit the accuracy and practicality of underwater three-dimensional reconstruction technology.

[0038] The main solution of the embodiment of the present application is: by emitting laser pulses and capturing reflected images, high-contrast feature information on the surface of the target object is obtained. Then, the pixel coordinates of the laser stripes are extracted from the images, and the three-dimensional spatial coordinates of the laser points are calculated using the camera internal parameter matrix. Based on these depth information, control instructions are generated to guide the underwater vehicle to perform an equidistant diameter circular motion to collect high-quality three-dimensional point cloud data at different poses, and the motion trajectory is dynamically adjusted to ensure the uniformity and accuracy of data collection. Finally, the three-dimensional point cloud data at each pose is fused to generate a complete target three-dimensional point cloud model, completing the three-dimensional reconstruction.

[0039] It should be noted that the execution subject of the embodiments of the present application can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a monocular line laser device, etc. that can implement the above functions. Hereinafter, taking the monocular line laser device as an example, the present embodiment and the following embodiments will be described.

[0040] Based on this, the embodiments of the present application provide a three-dimensional reconstruction method for an underwater environment. Refer to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the three-dimensional reconstruction method for the underwater environment of the present application.

[0041] In this embodiment, the three-dimensional reconstruction method for the underwater environment includes steps S10 to S50:

[0042] Step S10: Emit laser light to a target object and collect a laser reflection image.

[0043] It should be noted that the target object refers to a specific object or area in the underwater environment that needs to be three-dimensionally reconstructed, which can be a seabed geological structure, a seabed pipeline, a dam crack, a sunken ship, a marine biological habitat, etc.

[0044] The laser reflection image refers to the image captured by the monocular camera when the light reflected back after the laser is emitted to the surface of the target object, which contains the information of the laser reflected from the surface of the target object. These information are presented in the form of an image, and the brightness or gray value of each pixel point reflects the intensity of the laser reflected from the surface of the target object at that point.

[0045] It can be understood that first, the monocular line laser device will start the laser emitter and emit a line of laser light to the target object underwater, and this line of laser light forms a bright light band or light spot on the surface of the target object. Secondly, when the laser pulse touches the surface of the target object, part of the light will be reflected back, and the monocular camera in the monocular line laser device will capture these reflected lights within a predetermined exposure time to form an image containing the light band or light spot, that is, the laser reflection image.

[0046] Step S20: Extract the pixel coordinates of the laser stripe from the laser reflection image.

[0047] It should be noted that the laser stripe refers to the light band formed after the laser emitter emits laser light to the surface of the target object and is reflected back to the monocular camera.

[0048] The pixel coordinates refer to the position of each pixel point in the image, which is represented by the indexes of rows and columns. In the laser reflection image, the laser stripe is composed of a series of pixel points. Extracting the pixel coordinates of the laser stripe is to determine the specific positions of these pixel points in the image.

[0049] It is understandable that, first, the collected laser reflection image is grayscaled to convert the color image into a grayscale image. Second, using the threshold segmentation method, a suitable grayscale threshold is set, and the pixel points in the image with grayscale values higher than this threshold are identified as part of the laser stripe, so as to effectively distinguish the laser stripe from other background information. Finally, each pixel point in the image is traversed, and the row and column positions of all pixel points belonging to the laser stripe, that is, their pixel coordinates, are recorded, and these coordinates will be used for subsequent depth calculation and three-dimensional reconstruction.

[0050] As an example, the step of extracting the pixel coordinates of the laser stripe from the laser reflection image includes: grayscaling the laser reflection image to obtain a grayscale distribution image; performing binary segmentation on the grayscale distribution image based on a preset laser spot intensity threshold to obtain a laser stripe mask region; scanning the laser stripe mask region row by row, and extracting the continuous pixel region with the largest grayscale value in each row as a candidate stripe segment; fitting the center line of the candidate stripe segment to obtain the center line coordinates of the laser stripe in the image; and extracting the pixel coordinates of the laser stripe at a preset sampling interval based on the center line coordinates.

[0051] The grayscale distribution image refers to the result after converting the original color laser reflection image into a grayscale image, where the value of each pixel point represents its brightness or grayscale level. The preset laser spot intensity threshold refers to a grayscale value standard set during the image processing to distinguish the laser spot from the background. For example, if the typical grayscale value range of the laser spot is between 200 - 255, and the grayscale value of the background is lower, then a threshold, such as 150, can be set, and the pixel points with grayscale values higher than 150 are regarded as laser spots.

[0052] The laser stripe mask region refers to the region in the image marked as the laser spot after binary segmentation. This region contains the pixel points of the laser spot, and the grayscale values of these points exceed the preset threshold.

[0053] The grayscale value refers to the brightness value of the pixel in the image, and the range usually ranges from 0 (black) to 255 (white). In a grayscale image, the higher the grayscale value, the brighter the pixel point. The candidate stripe segment refers to the continuous pixel region with the largest grayscale value found during the row-by-row scan within the laser stripe mask region. These regions are considered to be part of the laser stripe because they have a higher grayscale value, reflecting the position of the laser spot.

[0054] Centerline fitting refers to performing mathematical processing on candidate stripe segments to find a straight line that best represents the center position of the stripe. This is usually achieved by calculating the average position of all candidate pixel points or using fitting algorithms such as the least squares method. Centerline coordinates refer to the coordinate points in the straight line equation obtained through centerline fitting, and these points represent the center position of the laser stripe in the image. The preset sampling interval refers to the interval at which a coordinate point is extracted at a certain distance along the centerline when extracting the pixel coordinates of the laser stripe.

[0055] First, the monocular line laser device converts the laser reflection image into a grayscale image. This is done to simplify the image processing process because the grayscale image only contains brightness information, which is convenient for subsequent binary segmentation operations. Second, the device performs binary processing on the grayscale image according to the preset laser spot intensity threshold, marking the pixel points with grayscale values higher than the threshold as laser stripes and those lower than the threshold as the background. This is done to separate the laser stripes from the image for subsequent precise extraction. Then, the device scans the laser stripe mask area row by row to find the continuous pixel area with the largest grayscale value in each row as the candidate stripe segment. These areas represent the projection of the laser on the image, and the position of the laser stripe can be initially located through this method. Next, centerline fitting is performed on each candidate stripe segment to calculate the centerline coordinates of each stripe. This is usually achieved by finding the average position of all pixels in the candidate stripe segment or using fitting techniques such as the least squares method. This is done to determine the center position of the laser stripe. Finally, based on the centerline coordinates, the device extracts the pixel coordinates of the laser stripe from the centerline at the preset sampling interval, such as every 5 pixels. These coordinates will be used to calculate the three-dimensional spatial coordinates of the laser point, which can reduce the data volume while retaining sufficient information for precise three-dimensional reconstruction.

[0056] Step S30: Calculate the three-dimensional spatial coordinates of the laser point based on the pixel coordinates and the camera internal parameter matrix.

[0057] It should be noted that the camera internal parameter matrix refers to the parameter matrix that describes the internal geometry and optical characteristics of the camera. It reflects the relationship between the pixel coordinates and the camera coordinate system in the camera imaging system. Specifically, the camera internal parameter matrix includes parameters such as focal length, principal point coordinates (optical center position), and pixel size. These parameters can help us accurately map the pixel points in the two-dimensional image to the three-dimensional space, providing a basis for subsequent three-dimensional reconstruction.

[0058] A laser point refers to the light point formed after the laser emitter emits light to the surface of the target object and is reflected back to the monocular camera. In this embodiment, the laser point is each specific position point on the laser stripe, and these points appear as bright pixel points in the laser reflection image.

[0059] Three-dimensional space coordinates refer to the position of a point in three-dimensional space, which is represented by three coordinate values (x, y, z). In this embodiment, the three-dimensional space coordinates refer to the positions of points on the surface of the target object in the camera coordinate system or the world coordinate system.

[0060] It can be understood that, first, according to the known camera internal parameter matrix, the pixel coordinates of the extracted laser stripe are converted into the normalized image plane coordinates in the camera coordinate system. Then, combined with the geometric relationship between the laser emission angle and the camera, the distance of each laser point relative to the camera, that is, the depth information, is calculated. Finally, the depth information is combined with the normalized image plane coordinates to obtain the three-dimensional space coordinates of each laser point in the camera coordinate system, thereby realizing the conversion from two-dimensional image coordinates to three-dimensional space coordinates and providing accurate data support for constructing the three-dimensional point cloud model of the target object.

[0061] Step S40: Generate a control command according to the depth information of the three-dimensional space coordinates, and send the control command to the underwater vehicle so that the underwater vehicle performs an equidistant diameter circular motion, collects three-dimensional point cloud data in different poses, and feeds back.

[0062] It should be noted that the depth information refers to the distance information of points on the surface of the target object relative to the camera, which is obtained by calculating the three-dimensional space coordinates of the laser points.

[0063] The control command refers to the command for guiding the movement of the underwater vehicle generated according to the depth information, which is generated based on the depth information on the surface of the target object, and the purpose is to enable the underwater vehicle to dynamically adjust its pose to achieve an equidistant diameter circular motion.

[0064] An underwater vehicle is a robot capable of autonomous operation underwater, including AUV (Autonomous Underwater Vehicle, untethered autonomous underwater vehicle) and ROV (Remotely Operated Vehicle, tethered remotely operated underwater vehicle). In this embodiment, the AUV is a platform equipped with a monocular line laser device, responsible for moving in the underwater environment and collecting laser image data of the target object. The AUV has autonomous navigation and motion control capabilities, and can adjust its motion trajectory and attitude according to the control command to ensure high-quality three-dimensional point cloud data can be collected in different poses.

[0065] Equidistant diameter circular motion refers to the circular motion of the AUV around the target object while maintaining a constant distance from the target object during the motion. This motion mode is to ensure that the AUV can uniformly collect the three-dimensional point cloud data of the target object at different angles and positions. The three-dimensional point cloud data refers to the three-dimensional coordinate information of each point on the surface of the target object collected by a laser scanning device, which is generated after processing the laser image data collected by a monocular line laser device. The three-dimensional coordinates of each laser point form a point in the point cloud, and the set of these points is the three-dimensional point cloud data.

[0066] Please refer to Figure 2 , Figure 2 Figure Figure 2 is a schematic diagram of the dynamic adjustment of the AUV based on depth information provided for the first embodiment of the underwater environment three-dimensional reconstruction method of this application. First, depth information is obtained through a laser image, and this information is input into the monocular line laser device to guide the AUV for target tracking. According to the results of target tracking, the monocular line laser device determines whether distance adjustment and equidistant circular motion are required to ensure that the AUV maintains a constant distance from the target object, thereby optimizing the integrity and accuracy of data collection. If the environmental conditions are met, the AUV will perform equidistant circular motion to collect omnidirectional data; if not, it will move along the original planned path. The whole process forms a closed-loop control system. By continuously adjusting the motion trajectory of the AUV, it is ensured that the three-dimensional point cloud data collected at different poses is both complete and accurate, thereby improving the accuracy and efficiency of three-dimensional reconstruction.

[0067] It can be understood that, first, the monocular line laser device calculates the real-time distance between the AUV and the target object based on the depth information of the three-dimensional spatial coordinates of the extracted laser points. Secondly, based on this distance information, the monocular line laser device generates control instructions that can adjust the motion trajectory of the AUV to keep it at a constant distance from the target object. Finally, after receiving the control instructions, the AUV performs equidistant diameter circular motion. During the motion, the monocular line laser device carried by the AUV continuously collects the three-dimensional point cloud data at different poses and feeds these data back to the monocular line laser device in real time for subsequent three-dimensional reconstruction.

[0068] As an example, the step of generating a control instruction based on the depth information of the three-dimensional space coordinates, sending the control instruction to an underwater vehicle to make the underwater vehicle perform a circular motion with an equidistant diameter, collecting three-dimensional point cloud data at different poses, and feeding back includes: determining the real-time distance between the target object and the underwater vehicle according to the depth information of the three-dimensional space coordinates; in the case where the real-time distance deviates from a preset distance threshold, determining a speed adjustment parameter and a direction adjustment parameter according to the real-time distance and the preset distance threshold; generating a control instruction according to the speed adjustment parameter and the direction adjustment parameter; sending the control instruction to the underwater vehicle to make the underwater vehicle perform a circular motion with an equidistant diameter, collect three-dimensional point cloud data at different poses, and feed back.

[0069] The preset distance threshold refers to the ideal distance range that should be maintained between the AUV and the target object during the underwater three-dimensional reconstruction process. This threshold is determined according to the size and shape of the target object and the required three-dimensional reconstruction accuracy.

[0070] The speed adjustment parameter refers to the parameter calculated according to the deviation between the real-time distance between the AUV and the target object and the preset distance threshold for adjusting the speed of the AUV. When the real-time distance deviates from the preset distance threshold, the monocular line laser device calculates the speed adjustment parameter according to the magnitude and direction of the deviation. These parameters determine the degree to which the AUV needs to accelerate or decelerate so that it can quickly return to the preset distance range.

[0071] The direction adjustment parameter refers to the parameter calculated according to the deviation between the real-time distance between the AUV and the target object and the preset distance threshold for adjusting the motion direction of the AUV. When the real-time distance deviates from the preset distance threshold, the system calculates the direction adjustment parameter according to the direction of the deviation. These parameters determine the direction angle that the AUV needs to adjust so that it can move along the correct trajectory and maintain a constant distance from the target object.

[0072] First, the monocular line laser device calculates the real-time distance between the current AUV and the target object through its built-in processing unit using the depth information in the three-dimensional spatial coordinates of the acquired laser points. This distance is determined by analyzing the depth values of each laser point, providing an accurate basis for subsequent motion adjustment. Second, when there is a deviation between this real-time distance and the preset distance threshold, the processing unit calculates the speed adjustment parameter and the direction adjustment parameter according to the specific value of the deviation and the pre-set algorithm. The speed adjustment parameter determines the magnitude of acceleration or deceleration required for the AUV, while the direction adjustment parameter indicates the direction in which the AUV needs to adjust its motion trajectory to ensure that the AUV can return to the preset circular motion trajectory with an equal distance diameter and maintain a constant distance from the target object, thus ensuring the uniformity and integrity of data collection. Finally, the processing unit generates specific control instructions based on these adjustment parameters, and these instructions are sent to the control system of the AUV through the communication link. After receiving the instructions, the AUV adjusts the rotation speed of its thrusters and the angle of its rudder to perform circular motion with an equal distance diameter. During the motion, the monocular line laser device continuously collects three-dimensional point cloud data in different poses and real-time feeds these data back to the processing unit through the communication system of the AUV for subsequent three-dimensional reconstruction and analysis.

[0073] Step S50: Fuse the three-dimensional point cloud data in different poses to obtain a target three-dimensional point cloud model and complete the three-dimensional reconstruction.

[0074] It should be noted that the target three-dimensional point cloud model refers to the complete three-dimensional point cloud representation of the target object obtained by fusing the three-dimensional point cloud data collected in different poses. This model contains the three-dimensional coordinate information of all points on the surface of the target object and can accurately reflect the shape, size, and surface characteristics of the target object.

[0075] It can be understood that first, multiple sets of three-dimensional point cloud data collected by the AUV in different poses are collected, and these data contain the three-dimensional coordinate information of each point on the surface of the target object. Then, using the point cloud registration algorithm, these scattered point cloud data are aligned and spliced. By calculating the spatial transformation relationship between the point clouds, they are unified into the same coordinate system to ensure seamless docking of the data collected in different poses. Finally, the aligned point cloud data are fused to remove duplicate points and noise points, generating a complete and accurate target three-dimensional point cloud model.

[0076] As an example, the step of fusing the 3D point cloud data in different poses to obtain a target 3D point cloud model and complete 3D reconstruction includes: obtaining the rotation matrix and translation vector of the underwater vehicle in different poses; according to the rotation matrix and the translation vector, converting each frame of the 3D point cloud data in different poses to the first frame coordinate system to obtain reference point cloud data; performing outlier filtering and multi-view registration on the reference point cloud data to obtain target point cloud data; superimposing and fusing the target point cloud data to obtain a target 3D point cloud model and complete 3D reconstruction.

[0077] The rotation matrix is a 3×3 orthogonal matrix used to describe the rotation state of the AUV in different poses. The rotation matrix is obtained through the AUV's attitude sensor (such as a gyroscope) or through a point cloud registration algorithm. It can rotate a point from one coordinate system to another. The rotation matrix ensures that the 3D point cloud data collected in different poses can be correctly aligned in space, providing a basis for subsequent point cloud fusion. The translation vector is a three-dimensional vector used to describe the position change of the AUV in different poses. The translation vector is obtained through the AUV's position sensor (such as GPS, acoustic positioning system) or through a point cloud registration algorithm. It can translate a point from one coordinate system to another. The translation vector combined with the rotation matrix can describe the spatial transformation of the AUV in different poses.

[0078] The first frame coordinate system refers to the coordinate system established when the AUV collects the first frame of 3D point cloud data. The first frame coordinate system serves as a reference coordinate system, and all subsequent collected point cloud data will be transformed into this coordinate system. The reference point cloud data refers to the data obtained by transforming the 3D point cloud data in different poses to the first frame coordinate system through the rotation matrix and translation vector. It is the point cloud data after spatial alignment. They are in the same coordinate system, providing a basis for subsequent point cloud fusion and processing. Outlier filtering is a data processing method used to remove abnormal points or noise points in the reference point cloud data.

[0079] Multi-view registration refers to the process of precisely aligning the reference point cloud data from different perspectives through the Iterative Closest Point (ICP) algorithm or other registration algorithms. The ICP algorithm iteratively finds the closest point pairs and calculates the optimal rotation and translation transformations to make the point clouds from different perspectives better fuse together. The target point cloud data refers to the point cloud data after outlier filtering and multi-view registration. It is high-quality point cloud data after optimization processing. They are aligned in space and the noise points are removed. Superimposing and fusing refers to the process of combining the target point cloud data to generate the final target 3D point cloud model.

[0080] The coordinate transformation formula is as follows:

[0081]

[0082] Among them, R i represents the rotation of the AUV, and t i represents the displacement of the AUV platform. (X qij , Y qij , Z qij ) are the three-dimensional coordinates of point j in different i-frame coordinate systems under the first-frame coordinate system. The first-frame coordinate system is established by collecting the first-frame data with a monocular line laser device. Through the external parameters (R i , t i ), the point cloud maps obtained by the AUV in different poses can be stitched in one coordinate system to obtain a complete point cloud map.

[0083] When the AUV platform carries devices to perform scanning and reconstruction of multiple targets, the point cloud coordinate systems of each target are inconsistent and cannot be fused in one coordinate system. Therefore, it is necessary to specify a world origin to fuse the point cloud information of different targets, but the rotation and translation pose information from the camera origin of the monocular line laser device to this point needs to be known. The transformation relationship from the point cloud coordinate system to the world coordinate system is as follows:

[0084]

[0085] Among them, R d is a 3x3 rotation matrix used to describe the rotation of coordinate system q relative to coordinate system w, and t d is a translation vector representing the translation of the origin of coordinate system q relative to the origin of coordinate system w. w represents the world coordinate system, q represents different targets, and (X wij , Y wij , Z wij ) are the three-dimensional coordinates of point j in different i-frame coordinate systems under the world coordinate system. In this embodiment, the complete coordinate transformation formula is:

[0086]

[0087] Among them, Z ij is the depth information of point (i, j) in the original coordinate system.

[0088] First, the monocular line laser device obtains the rotation matrix and translation vector of the AUV at different poses by its built-in sensors or communicating with the AUV's navigation system. These data are used to describe the specific position and attitude changes of the AUV in space. Then, the device uses these rotation matrices and translation vectors to perform a spatial transformation on each frame of three-dimensional point cloud data collected at different poses, converting it into the coordinate system of the first frame to obtain the reference point cloud data. This process is achieved through coordinate transformation formulas to ensure that all point cloud data are aligned in the same coordinate system. Next, outlier filtering is performed on the reference point cloud data. By statistical analysis or distance threshold methods, abnormal points that are too far from surrounding points or do not conform to the overall distribution law are identified and removed, improving the purity and accuracy of the data. Then, multi-view registration is carried out. Using methods such as ICP, the reference point cloud data from different views are accurately aligned. The ICP algorithm iteratively finds the closest point pairs and calculates the optimal rotation and translation transformations, enabling the point cloud data from each view to be better fused together and further improving the alignment accuracy. Finally, the target point cloud data that has undergone outlier filtering and multi-view registration are superimposed and fused. Through simple point cloud merging or voxel grid fusion and other methods, all target point cloud data are integrated into a unified point cloud model to generate a complete target three-dimensional point cloud model, thus completing the three-dimensional reconstruction and providing accurate three-dimensional data support for subsequent analysis and applications.

[0089] This embodiment provides a method for three-dimensional reconstruction of an underwater environment. First, the monocular line laser device emits laser light towards the target object and collects the laser reflection image. This process obtains the high-contrast feature information on the surface of the target object by emitting laser pulses and capturing the reflected light, providing the basic data for subsequent three-dimensional reconstruction. Next, the pixel coordinates of the laser stripe are extracted from the collected laser reflection image. The position of the laser stripe is extracted through image processing algorithms, providing accurate input data for subsequent depth calculation. Then, based on the extracted pixel coordinates and the camera internal parameter matrix, the three-dimensional spatial coordinates of the laser points are calculated, mapping the feature points in the two-dimensional image into the three-dimensional space. After that, control instructions are generated according to the depth information of the laser points and sent to the AUV, enabling it to perform an equidistant diameter circular motion, collect three-dimensional point cloud data at different poses and feedback. By dynamically adjusting the motion trajectory of the AUV, high-quality three-dimensional point cloud data are uniformly collected at different angles and positions, reducing the data acquisition error caused by distance changes. Finally, the three-dimensional point cloud data at different poses are fused to generate a target three-dimensional point cloud model, thus completing the three-dimensional reconstruction. This process not only improves the integrity and accuracy of the three-dimensional reconstruction but also provides high-quality three-dimensional data support for subsequent analysis and applications. This embodiment can efficiently, accurately and low-costly achieve three-dimensional reconstruction of the underwater environment, providing strong technical support for underwater exploration and scientific research.

[0090] Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , Figure 3 which is a schematic flowchart of the second embodiment of the underwater environment three-dimensional reconstruction method of this application. The step S30 of the underwater environment three-dimensional reconstruction method includes steps S31 to S34:

[0091] Step S31, obtain the camera internal parameter matrix according to the Zhang Zhengyou calibration method.

[0092] It should be noted that the Zhang Zhengyou calibration method is a widely used camera calibration method. By using a known geometric pattern (usually a checkerboard or dot pattern) as a calibration board and combining image processing and mathematical optimization algorithms, the internal parameter matrix of the camera can be accurately calculated.

[0093] It can be understood that, first, the camera of the monocular line laser device takes multiple images of the calibration board from different angles and positions to ensure that the calibration board presents different postures and positions in the images, so as to obtain rich perspective information and provide sufficient data for subsequent calibration. Secondly, the device detects the feature points on the calibration board in each image through image processing algorithms, such as the corner points of the checkerboard or the center points of the dot pattern, and records the pixel coordinates of these feature points. These feature points are the key bridges connecting the image coordinate system and the real-world coordinate system. Finally, the device uses the corresponding relationship between the detected pixel coordinates of the feature points and the known world coordinates, and calculates the internal parameter matrix of the camera through optimization algorithms such as the least squares method, including parameters such as focal length, principal point coordinates, and pixel size. These internal parameter matrices can accurately describe the imaging characteristics of the camera and provide an accurate parameter basis for subsequent three-dimensional reconstruction, ensuring the accuracy and reliability of three-dimensional reconstruction.

[0094] Step S32, perform oblique laser triangulation ranging on the pixel coordinates based on the camera internal parameter matrix to obtain the depth information corresponding to the pixel coordinates.

[0095] It should be noted that oblique laser triangulation ranging is a method of measuring the depth information of points on the surface of a target object by using the triangular relationship formed among a laser emitter, the surface of the target object, and the camera. The depth information refers to the distance information of each point on the surface of the target object relative to the camera.

[0096] It can be understood that, first, the pixel coordinates extracted are converted into normalized image plane coordinates in the camera coordinate system by using the camera internal parameter matrix. This step is a necessary conversion for mapping two-dimensional image information to three-dimensional space. Then, combined with the geometric relationship between the laser emitter and the camera, the distance from the point on the surface of the target object to the camera optical center, that is, the depth information, is calculated through the principle of triangulation.

[0097] As an example, the step of performing oblique laser triangulation ranging on the pixel coordinates based on the camera internal parameter matrix to obtain the depth information corresponding to the pixel coordinates includes: determining the angle between the laser emission direction and the camera optical axis according to the relative position parameters of the camera and the laser; based on the camera internal parameter matrix, inverse-projecting the pixel coordinates into the camera coordinate system to obtain the imaging light path direction vector; constructing a laser light path direction vector according to the angle, and forming a triangulation ranging geometric model with the imaging light path direction vector; and obtaining the depth information corresponding to the pixel coordinates according to the triangulation ranging geometric model.

[0098] The monocular line laser device includes a monocular camera and a line laser. The relative position parameters refer to the installation position relationship between the monocular camera and the line laser on the device, including information such as the distance, angle, and direction between them.

[0099] The laser emission direction refers to the propagation direction of the laser in space after it departs from the emitter, and is determined by the laser plane equation parameters and the relative position parameters. Specifically, the laser emission direction starts from the focus of the laser emitter and propagates along the normal direction of the laser plane.

[0100] The camera optical axis refers to the straight line starting from the camera optical center and passing through the camera principal point, and is usually perpendicular to the camera imaging plane. The camera optical axis is part of the camera internal parameter matrix, which is used to describe the imaging direction of the camera and can be determined by the focal length and principal point coordinates in the camera internal parameter matrix.

[0101] The camera coordinate system refers to the coordinate system fixedly connected to the camera itself, which defines the position and direction of the camera in space. This coordinate system usually takes the camera optical center (i.e., the optical center of the camera lens) as the origin, and its X, Y, and Z axes are respectively perpendicular to the camera imaging plane and parallel to the sides of the imaging plane.

[0102] The imaging light path direction vector refers to the vector starting from the camera optical center, passing through a certain pixel point on the image, and extending to infinity. This vector represents the direction of the camera imaging light path and is used to determine the direction of points in the camera coordinate system.

[0103] The laser light path direction vector refers to the vector starting from the laser emitter and pointing to a certain point on the target object. This vector represents the direction of laser emission and is used to determine the propagation path of the laser in space. The construction of the laser light path direction vector is based on the angle between the laser emission direction and the camera optical axis, and this angle is determined according to the relative position parameters of the camera and the laser.

[0104] The triangulation geometry model is a geometric model used to measure the depth of a point on the surface of a target object, constructed using the direction vectors of the laser and imaging light paths. This model determines the location of a point on the target surface by analyzing the intersection of the laser and imaging light paths. In this model, the laser and imaging light path direction vectors intersect at the same point on the target object in three-dimensional space.

[0105] First, the monocular line laser device calculates the angle between the laser emission direction and the camera optical axis based on the relative position parameters of the camera and the laser. This step is achieved through trigonometric functions. For example, if the relative position parameters give the installation angle θ between the laser and the camera optical axis, then the angle θ is directly used in subsequent calculations to ensure the accuracy of the laser optical path direction. Secondly, the device uses the focal length (fx, fy) and principal point coordinates (cx, cy) in the camera intrinsic parameter matrix to convert the pixel coordinates (u, v) into normalized image plane coordinates (x, y). The specific formula is:

[0106]

[0107] Then, an imaging light path direction vector (x, y, 1) is generated, representing the direction of the light from the camera's optical center through the pixel point. This step ensures that the pixel coordinates can be mapped into three-dimensional space. The device then constructs the laser light path direction vector based on the angle θ. Assuming that the laser emission direction is in the XOZ plane of the camera coordinate system, the direction vector is (sinθ, 0, cosθ). Together with the imaging light path direction vector, a triangulation ranging geometry model is formed. This model determines the position of the target point by combining the parametric equations of the laser light path and the imaging light path to find their intersection in three-dimensional space. Finally, by solving the set of equations in the triangulation ranging geometry model, the depth information corresponding to each pixel coordinate is obtained, that is, the distance from the surface point of the target object to the camera. This depth information provides accurate data support for subsequent three-dimensional reconstruction.

[0108] As an example, the steps of constructing a laser light path direction vector based on the angle and forming a triangulated ranging geometric model with the imaging light path direction vector include: constructing a laser light path direction vector based on the angle; establishing an intersection constraint condition between the laser light path direction vector and the imaging light path direction vector in three-dimensional space; and generating a triangulated ranging geometric model of the laser light path and the camera imaging light path based on the laser light path direction vector, the imaging light path direction vector and the intersection constraint condition.

[0109] The laser optical path direction vector refers to the propagation direction of the laser in space, which is constructed based on the angle between the laser emission direction and the camera optical axis. Specifically, if the angle is θ, the direction vector can be expressed as (sinθ, 0, cosθ), assuming that the laser emission direction is in the XOZ plane of the camera coordinate system. This vector describes the propagation path of the laser in space after leaving the emitter.

[0110] The intersection constraint condition means that the laser optical path direction vector and the imaging optical path direction vector must intersect at the same point in three-dimensional space. This condition ensures that the intersection points of the laser optical path and the imaging optical path on the surface of the target object are consistent, thus guaranteeing the accuracy of the measured depth information. Specifically, the point P = P0 + t * laser optical path direction vector on the laser optical path and the point Q = Q0 + s * imaging optical path direction vector on the camera imaging optical path must intersect at the same point on the surface of the target object, where P0 is the coordinate of the laser emission point in the camera coordinate system and Q0 is the coordinate of the camera optical center.

[0111] First, the monocular line laser device constructs the laser optical path direction vector according to the angle θ between the laser emission direction and the camera optical axis. Specifically, by substituting θ into the trigonometric functions, the direction vector (sinθ, 0, cosθ) is obtained, assuming that the laser emission direction is in the XOZ plane of the camera coordinate system. This step ensures the accuracy of the laser optical path direction. Secondly, the device establishes the intersection constraint condition of the laser optical path direction vector and the imaging optical path direction vector in three-dimensional space. By setting the point P = P0 + t·(sinθ, 0, cosθ) on the laser optical path and the point Q = Q0 + s·(x, y, 1) on the camera imaging optical path, where P0 is the laser emission point, Q0 is the camera optical center, and t and s are parameters, and requiring P and Q to be equal, that is, P0 + t·(sinθ, 0, cosθ) = Q0 + s·(x, y, 1). This step ensures that the two optical paths intersect at the target point, thus guaranteeing the accuracy of the depth information.

[0112] Finally, the device generates a triangulation ranging geometric model of the laser optical path and the camera imaging optical path according to the laser optical path direction vector, the imaging optical path direction vector, and the intersection constraint condition:

[0113] P 0x +t·d lx =Q 0x +s·d cx

[0114] P 0y +t·d ly =Q 0y +s·d cy

[0115] P 0z +t·d lz =Q 0z+s·d cz

[0116] where P0 = (P 0x , P 0y , P 0z ) refers to the coordinates of the laser emission point in the camera coordinate system, representing the starting point of the laser optical path; d l = (d lx , d ly , d lz ) refers to the laser optical path direction vector, representing the propagation direction of the laser in space; t is the parameter along the laser optical path, representing the distance along the laser optical path direction starting from the laser emission point; Q0 = (Q 0x , Q 0y , Q 0z ) refers to the coordinates of the camera optical center in the camera coordinate system, usually the origin; d c = (d cx , d cy , d cz ) refers to the imaging optical path direction vector, representing the light ray direction from the camera optical center passing through a specific pixel point on the image; s is the parameter along the imaging optical path, representing the distance along the imaging optical path direction starting from the camera optical center; P(t) = (P x , P y , P z ) refers to the point on the laser optical path, representing the position of the laser optical path at parameter t; Q(s) = (Q x , Q y , Q z ) refers to the point on the imaging optical path, representing the position of the imaging optical path at parameter s.

[0117] The parameters t and s can be solved by linear algebra methods, and then substituted into any one of the optical path equations to obtain the three-dimensional coordinates of the intersection point, that is, the depth information of the surface point of the target object.

[0118] Step S33, construct the three-dimensional point correspondence between the pixel coordinate system and the camera coordinate system according to the camera internal parameter matrix and the depth information.

[0119] It should be noted that the three-dimensional point correspondence refers to the mapping relationship between each pixel point in the pixel coordinate system and the three-dimensional space point in the camera coordinate system, in the following form:

[0120]

[0121] where (u, v) are the pixel coordinates, i represents the movement process from the AUV collecting the i-th frame of data to the (i + 1)-th frame of data, and j represents the laser pixel point in the i-th frame of image data. The matrix composed of intermediate quantities is the internal parameter matrix K of the camera, Z ijDenote the laser pixel point j of the i-th frame of laser image data, (X ij , Y ij , Z ij ) represents the three-dimensional point cloud corresponding to each pixel point j under the i-th frame of data, that is, the three-dimensional coordinates of point j in the i-th frame coordinate system.

[0122] It can be understood that, first, the monocular line laser device uses the camera internal parameter matrix to convert the pixel coordinates of each pixel point into normalized image plane coordinates, providing a basis for subsequent depth calculation. Secondly, the device combines the depth information to convert the normalized image plane coordinates into three-dimensional coordinates in the camera coordinate system. This step is to map the points in the two-dimensional image into the three-dimensional space, so as to obtain the actual spatial position corresponding to each pixel point. Finally, through these conversions, the device constructs the three-dimensional point correspondence between the pixel coordinate system and the camera coordinate system.

[0123] Step S34, calculate the three-dimensional spatial coordinates of the laser point according to the three-dimensional point correspondence and the pixel coordinates.

[0124] It can be understood that the monocular line laser device substitutes the pixel coordinates into the three-dimensional point correspondence to obtain the three-dimensional spatial coordinates.

[0125] Please refer to Figure 4 , Figure 4 , which is the schematic diagram of the coordinate conversion process provided in the second embodiment of the three-dimensional reconstruction method for the underwater environment of this application. First, the monocular line laser device obtains the laser coordinates, camera internal parameters, and laser point depth information through coordinate extraction, Zhang Zhengyou calibration method, and ranging model. Then, use the back-projection formula to convert this information into single-frame point cloud information. This formula is based on the camera internal parameter matrix and depth information to convert pixel coordinates into three-dimensional coordinates in the camera coordinate system. Then, obtain the corresponding AUV pose according to the timestamp, that is, the specific position and direction of the AUV underwater. With the AUV pose, the single-frame point cloud data can be spatially transformed according to the pose information and stitched into the complete point cloud of the target object. Finally, through this complete point cloud information, not only can the three-dimensional model of the target object be reconstructed, but also the point cloud information of the surrounding environment can be obtained, providing important data support for the comprehensive understanding of the underwater environment.

[0126] In this embodiment, the camera internal parameter matrix is obtained through the Zhang-Zhang calibration method, ensuring an accurate description of the internal optical characteristics of the camera. Then, based on the camera internal parameter matrix, oblique laser triangulation ranging is performed to accurately measure the depth information of each point on the surface of the target object, providing the necessary data for generating a three-dimensional point cloud. Next, the three-dimensional point correspondence between the pixel coordinate system and the camera coordinate system is constructed using the camera internal parameter matrix and the depth information, converting the two-dimensional image information into three-dimensional space information, and providing accurate point cloud data for three-dimensional reconstruction. Finally, the three-dimensional spatial coordinates of the laser points are calculated according to the three-dimensional point correspondence and the pixel coordinates, generating a complete three-dimensional point cloud on the surface of the target object, providing accurate geometric information for subsequent three-dimensional reconstruction and analysis. This series of steps makes the three-dimensional reconstruction process of the underwater environment more efficient, accurate, and cost-effective, providing strong technical support for underwater exploration and scientific research.

[0127] Exemplarily, to facilitate understanding of the implementation process of the underwater environment three-dimensional reconstruction method obtained by combining this embodiment with the above-mentioned Embodiment 1, please refer to Figure 5 , Figure 5 which provides a schematic diagram of the brief process of an underwater environment three-dimensional reconstruction method. Specifically:

[0128] This figure shows the complete process of an AUV carrying a monocular line laser device for underwater three-dimensional reconstruction. First, the monocular line laser device carried by the AUV emits laser light to the target object and obtains a laser image, and then the coordinates of the laser stripe are extracted from the image. Using the camera internal parameters and depth information, these coordinates are converted into normalized image plane coordinates, and then the three-dimensional spatial coordinates of each laser point are calculated through the back-projection formula. Next, the attitude of the AUV is dynamically adjusted according to the ranging information, and an equidistant diameter circular motion is executed to collect images in different poses. At each pose, the above process is repeated to obtain single-frame point cloud information, and then according to the time stamp and the AUV pose information, the point clouds of each frame are stitched together to form a complete three-dimensional point cloud model of the target object. At the same time, this process also collects the three-dimensional point cloud information of the environment to provide support for a comprehensive understanding of the underwater environment. This process realizes an accurate conversion from image data to a three-dimensional model and is a key step in high-precision underwater three-dimensional reconstruction.

[0129] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the underwater environment three-dimensional reconstruction method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0130] This application also provides an underwater environment three-dimensional reconstruction device. Please refer to Figure 6 , the underwater environment three-dimensional reconstruction device includes:

[0131] A laser emission module 10 for emitting laser light to the target object and collecting a laser reflection image;

[0132] A coordinate extraction module 20, configured to extract the pixel coordinates of the laser stripe from the laser reflection image;

[0133] A coordinate calculation module 30, configured to calculate the three-dimensional spatial coordinates of the laser point based on the pixel coordinates and the camera internal parameter matrix;

[0134] A collaborative control module 40, configured to generate a control instruction according to the depth information of the three-dimensional spatial coordinates, and send the control instruction to the underwater vehicle to enable the underwater vehicle to perform an equidistant diameter circular motion, collect three-dimensional point cloud data in different poses, and give feedback;

[0135] A fusion modeling module 50, configured to fuse the three-dimensional point cloud data in different poses to obtain a target three-dimensional point cloud model and complete three-dimensional reconstruction.

[0136] In one embodiment, the coordinate calculation module 30 is further configured to obtain the camera internal parameter matrix according to the Zhang Zhengyou calibration method; perform oblique laser triangulation ranging on the pixel coordinates based on the camera internal parameter matrix to obtain the depth information corresponding to the pixel coordinates; construct a three-dimensional point correspondence relationship between the pixel coordinate system and the camera coordinate system according to the camera internal parameter matrix and the depth information; calculate the three-dimensional spatial coordinates of the laser point according to the three-dimensional point correspondence relationship and the pixel coordinates.

[0137] In one embodiment, the coordinate calculation module 30 is further configured to determine the angle between the laser emission direction and the camera optical axis according to the relative position parameters of the camera and the laser; based on the camera internal parameter matrix, inverse-project the pixel coordinates to the camera coordinate system to obtain the imaging optical path direction vector; construct a laser optical path direction vector according to the angle, and form a triangulation ranging geometric model with the imaging optical path direction vector; obtain the depth information corresponding to the pixel coordinates according to the triangulation ranging geometric model.

[0138] In one embodiment, the coordinate calculation module 30 is further configured to construct a laser optical path direction vector according to the angle; establish an intersection constraint condition of the laser optical path direction vector and the imaging optical path direction vector in three-dimensional space; generate a triangulation ranging geometric model of the laser optical path and the camera imaging optical path according to the laser optical path direction vector, the imaging optical path direction vector, and the intersection constraint condition.

[0139] In one embodiment, the coordinate extraction module 20 is further configured to perform grayscale processing on the laser reflection image to obtain a grayscale distribution image; perform binary segmentation on the grayscale distribution image based on a preset laser spot intensity threshold to obtain a laser stripe mask area; scan the laser stripe mask area row by row, and extract the continuous pixel area with the largest grayscale value in each row as a candidate stripe segment; perform centerline fitting on the candidate stripe segment to obtain the centerline coordinates of the laser stripe in the image; and extract the pixel coordinates of the laser stripe at a preset sampling interval based on the centerline coordinates.

[0140] In one embodiment, the collaborative control module 40 is further configured to determine the real-time distance between the target object and the underwater vehicle according to the depth information of the three-dimensional space coordinates; in the case where the real-time distance deviates from a preset distance threshold, determine a speed adjustment parameter and a direction adjustment parameter according to the real-time distance and the preset distance threshold; generate a control instruction according to the speed adjustment parameter and the direction adjustment parameter; and send the control instruction to the underwater vehicle, so that the underwater vehicle performs an equidistant diameter circular motion, collects three-dimensional point cloud data in different poses, and feeds back.

[0141] In one embodiment, the fusion modeling module 50 is further configured to obtain the rotation matrix and translation vector of the underwater vehicle in different poses; convert each frame of the three-dimensional point cloud data in different poses to the first frame coordinate system according to the rotation matrix and the translation vector to obtain reference point cloud data; perform outlier filtering and multi-view registration on the reference point cloud data to obtain target point cloud data; and superimpose and fuse the target point cloud data to obtain a target three-dimensional point cloud model, thereby completing three-dimensional reconstruction.

[0142] The underwater environment three-dimensional reconstruction device provided by the present application adopts the underwater environment three-dimensional reconstruction method in the above embodiment, and can solve the technical problem of how to efficiently, accurately and low-costly realize the three-dimensional reconstruction of the underwater environment. Compared with the prior art, the beneficial effects of the underwater environment three-dimensional reconstruction device provided by the present application are the same as those of the underwater environment three-dimensional reconstruction method provided by the above embodiment, and other technical features in the underwater environment three-dimensional reconstruction device are the same as those disclosed in the above embodiment method, which will not be elaborated here.

[0143] The present application provides an underwater environment three-dimensional reconstruction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the underwater environment three-dimensional reconstruction method in the first embodiment above.

[0144] Next, refer to Figure 7, which shows a schematic structural diagram of an underwater environment three-dimensional reconstruction device suitable for implementing the embodiments of the present application. The underwater environment three-dimensional reconstruction device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The shown underwater environment three-dimensional reconstruction device is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0145] As Figure 7 shown, the underwater environment three-dimensional reconstruction device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 1002 or a program loaded from a storage device 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the underwater environment three-dimensional reconstruction device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, an LCD (Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the underwater environment three-dimensional reconstruction device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an underwater environment three-dimensional reconstruction device with various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems may be alternatively implemented or had.

[0146] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0147] The three-dimensional underwater environment reconstruction device provided by the present application adopts the three-dimensional underwater environment reconstruction method in the above embodiments, and can solve the technical problem of how to efficiently, accurately and low-costly realize the three-dimensional reconstruction of the underwater environment. Compared with the prior art, the beneficial effects of the three-dimensional underwater environment reconstruction device provided by the present application are the same as those of the three-dimensional underwater environment reconstruction method provided by the above embodiments, and other technical features in the three-dimensional underwater environment reconstruction device are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0148] It should be understood that each part disclosed in the present application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0149] As mentioned above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0150] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the three-dimensional underwater environment reconstruction method in the above embodiments.

[0151] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or flash memory), optical fibers, CD-ROM (Compact Disc - Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0152] The above computer-readable storage medium can be included in the three-dimensional reconstruction device for underwater environment; it can also exist independently without being assembled into the three-dimensional reconstruction device for underwater environment.

[0153] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by the three-dimensional reconstruction device for underwater environment, the three-dimensional reconstruction device for underwater environment is caused to: emit laser light to a target object and collect a laser reflection image; extract the pixel coordinates of the laser stripe from the laser reflection image; calculate the three-dimensional spatial coordinates of the laser points based on the pixel coordinates and the camera internal parameter matrix; generate a control instruction according to the depth information of the three-dimensional spatial coordinates, and send the control instruction to an underwater vehicle to cause the underwater vehicle to perform an equidistant diameter circular motion and collect three-dimensional point cloud data at different poses and feedback; fuse the three-dimensional point cloud data at different poses to obtain a target three-dimensional point cloud model and complete the three-dimensional reconstruction.

[0154] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a LAN (Local Area Network) or a WAN (Wide Area Network), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0155] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and this module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0156] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0157] The readable storage medium provided in this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for performing the above-mentioned three-dimensional reconstruction method of the underwater environment, and can solve the technical problem of how to efficiently, accurately, and low-costly implement the three-dimensional reconstruction of the underwater environment. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the three-dimensional reconstruction method of the underwater environment provided in the above embodiments, and will not be elaborated here.

[0158] The present application also provides a computer program product, including a computer program which, when executed by a processor, implements the steps of the three-dimensional reconstruction method for an underwater environment as described above.

[0159] The computer program product provided by the present application can solve the technical problem of how to efficiently, accurately and at low cost implement the three-dimensional reconstruction of an underwater environment. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the three-dimensional reconstruction method for an underwater environment provided in the above embodiments, and will not be elaborated herein.

[0160] The above are only partial embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or any direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A three-dimensional reconstruction method for an underwater environment, characterized in that, The method includes: Emitting a laser to a target object and acquiring a laser reflection image; Extracting the pixel coordinates of the laser stripe from the laser reflection image; Calculating the three-dimensional spatial coordinates of the laser point based on the pixel coordinates and the camera intrinsic matrix; Generating a control instruction according to the depth information of the three-dimensional spatial coordinates, and sending the control instruction to an underwater vehicle to make the underwater vehicle perform an equidistant diameter circular motion, acquire three-dimensional point cloud data in different poses and feedback; Fusing the three-dimensional point cloud data in different poses to obtain a target three-dimensional point cloud model and completing three-dimensional reconstruction.

2. The method according to claim 1, characterized in that The step of calculating the three-dimensional spatial coordinates of the laser point based on the pixel coordinates and the camera intrinsic matrix includes: Obtaining the camera intrinsic matrix according to the Zhang Zhengyou calibration method; Performing oblique laser triangulation ranging on the pixel coordinates based on the camera intrinsic matrix to obtain the depth information corresponding to the pixel coordinates; Constructing a three-dimensional point correspondence relationship between the pixel coordinate system and the camera coordinate system according to the camera intrinsic matrix and the depth information; Calculating the three-dimensional spatial coordinates of the laser point according to the three-dimensional point correspondence relationship and the pixel coordinates.

3. The method according to claim 2, characterized in that, The step of performing oblique laser triangulation ranging on the pixel coordinates based on the camera intrinsic matrix to obtain the depth information corresponding to the pixel coordinates includes: Determining the angle between the laser emission direction and the camera optical axis according to the relative position parameters of the camera and the laser; Based on the camera intrinsic matrix, inverse-projecting the pixel coordinates to the camera coordinate system to obtain the imaging optical path direction vector; According to the angle, constructing a laser optical path direction vector and forming a triangulation ranging geometric model with the imaging optical path direction vector; Obtaining the depth information corresponding to the pixel coordinates according to the triangulation ranging geometric model.

4. The method according to claim 3, characterized in that The step of constructing a laser optical path direction vector according to the angle and forming a triangulation ranging geometric model with the imaging optical path direction vector includes: Constructing a laser optical path direction vector according to the angle; Establishing the intersection constraint condition of the laser optical path direction vector and the imaging optical path direction vector in three-dimensional space; Generating a triangulation ranging geometric model of the laser optical path and the camera imaging optical path according to the laser optical path direction vector, the imaging optical path direction vector and the intersection constraint condition.

5. The method according to claim 1, wherein The step of extracting the pixel coordinates of the laser stripe from the laser reflection image includes: Performing grayscale processing on the laser reflection image to obtain a grayscale distribution image; Performing binary segmentation on the grayscale distribution image based on a preset laser spot intensity threshold to obtain a laser stripe mask region; Scanning the laser stripe mask region row by row, and extracting the continuous pixel region with the largest grayscale value in each row as a candidate stripe segment; Performing center line fitting on the candidate stripe segment to obtain the center line coordinates of the laser stripe in the image; Extracting the pixel coordinates of the laser stripe at a preset sampling interval based on the center line coordinates.

6. The method according to claim 1, wherein The step of generating a control instruction according to the depth information of the three-dimensional spatial coordinates, and sending the control instruction to an underwater vehicle to make the underwater vehicle perform an equidistant diameter circular motion, acquire three-dimensional point cloud data in different poses and feedback includes: Determine the real-time distance between the target object and the underwater vehicle according to the depth information of the three-dimensional space coordinates; In the case where the real-time distance deviates from the preset distance threshold, determine the speed adjustment parameter and the direction adjustment parameter according to the real-time distance and the preset distance threshold; Generate a control instruction according to the speed adjustment parameter and the direction adjustment parameter; Send the control instruction to the underwater vehicle, so that the underwater vehicle performs an equidistant diameter circular motion, collects three-dimensional point cloud data in different poses, and feeds back.

7. The method according to any one of claims 1 to 6, characterized in that, The step of fusing the three-dimensional point cloud data in different poses to obtain a target three-dimensional point cloud model and complete three-dimensional reconstruction includes: Obtain the rotation matrix and translation vector of the underwater vehicle in different poses; According to the rotation matrix and the translation vector, convert each frame of the three-dimensional point cloud data in different poses to the first frame coordinate system to obtain reference point cloud data; Perform outlier filtering and multi-view registration on the reference point cloud data to obtain target point cloud data; Overlay and fuse the target point cloud data to obtain a target three-dimensional point cloud model and complete three-dimensional reconstruction.

8. A three-dimensional reconstruction device for an underwater environment, characterized in that, The device includes: A laser emission module for emitting laser to a target object and collecting a laser reflection image; A coordinate extraction module for extracting the pixel coordinates of the laser stripe from the laser reflection image; A coordinate calculation module for calculating the three-dimensional space coordinates of the laser point based on the pixel coordinates and the camera internal parameter matrix; A cooperative control module for generating a control instruction according to the depth information of the three-dimensional space coordinates, and sending the control instruction to the underwater vehicle so that the underwater vehicle performs an equidistant diameter circular motion, collects three-dimensional point cloud data in different poses, and feeds back; A fusion modeling module for fusing the three-dimensional point cloud data in different poses to obtain a target three-dimensional point cloud model and complete three-dimensional reconstruction.

9. A three-dimensional underwater environment reconstruction device, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the underwater environment three-dimensional reconstruction method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, it implements the steps of the underwater environment three-dimensional reconstruction method according to any one of claims 1 to 7.

Images