Underwater target three-dimensional reconstruction method and device

Abstract

The invention discloses an underwater target three-dimensional reconstruction method and device, and relates to the technical field of underwater measurement. The method comprises the following steps: respectively collecting a laser stripe image and a high-order texture image reflected by an underwater target through a point cloud camera and a texture camera; performing point cloud reconstruction operation on the laser stripe image to obtain a laser point cloud gridding model; performing library construction operation on the high-order texture image to obtain texture features and a point cloud library; and matching the laser point cloud gridding model with the texture features and the point cloud library, and performing texture mapping and coloring rendering on the laser point cloud gridding model based on a matching result to generate an underwater target three-dimensional reconstruction image with high-order texture information. According to the scheme, the collected laser stripe image reflected by the underwater target and the high-order texture image of the underwater target are fused through a corresponding algorithm process, three-dimensional reconstruction of high-order texture coloring of the underwater target is achieved, and the three-dimensional reconstruction image of the underwater target with high precision and rich image information can be obtained.

Description

Technical Field

[0001] The present invention relates to the field of underwater measurement technology, and in particular to a method and a device for three-dimensional reconstruction of an underwater target. Background Art

[0002] Three-dimensional reconstruction of underwater target shapes and stereoscopic spatial reconstruction are urgent technical challenges to address diverse underwater scenarios, including detailed terrain mapping in seabed exploration, feature target identification in underwater search and rescue, equipment shape reconstruction in marine engineering maintenance, and object size measurement in marine ranching. These technologies enable 3D scanning of underwater targets (seabed terrain, search and rescue targets, engineering equipment, aquaculture objects, etc.) in real-world ocean scenarios to obtain depth information. The scanned pose data is then overlaid to convert the information into 3D information. Combined with deep learning and data twin technologies, underwater virtual reality simulation is achieved, providing visual, panoramic, stereoscopic, and real-time intelligent perception technology for the utilization and development of marine resources.

[0003] Currently, underwater target shape 3D reconstruction and stereoscopic space reconstruction technologies primarily include underwater acoustic 3D measurement and underwater optical 3D measurement. Underwater acoustic 3D measurement primarily involves the hardware forms of a 2D imaging sonar vertical array and a 3D imaging sonar. These methods offer the advantages of a longer range, but are limited by the inability to measure at close range and low measurement accuracy. Underwater optical 3D measurement primarily involves the hardware forms of underwater binocular cameras and underwater laser scanners. The underwater range of binocular cameras is limited by water turbidity, and obtaining a smaller depth estimation error requires a larger distance between the imaging centers of the two cameras. Underwater laser scanners offer advantages such as a moderate range, high signal resolution, high measurement accuracy, and a low computational effort for image data, making them suitable for near-field, refined 3D measurement scenarios underwater.

[0004] However, existing underwater laser scanners mainly use single-line lasers or parallel linear array lasers as signal sources. When the underwater target being measured has a stepped structure and the stepped structure has an interface parallel to the direction of the linear laser, the step depth of this stepped structure cannot be measured. In other words, underwater laser scanners using single-line lasers or parallel linear array lasers as signal sources have a measurement blind spot when measuring stepped structures. In addition, existing underwater laser scanners only perform three-dimensional reconstruction of the collected laser point cloud. Due to the above defects, the final generated three-dimensional reconstructed image of the underwater target is not accurate and lacks sufficient image information, making it impossible to truly restore the characteristics of the underwater target. Summary of the Invention

[0005] The main purpose of the present invention is to propose a three-dimensional reconstruction method for underwater targets, aiming to solve the technical problem that existing underwater laser scanners have measurement blind spots when measuring underwater stepped structures, and only perform three-dimensional reconstruction on the collected laser point clouds, resulting in the final generated three-dimensional reconstructed image of the underwater target having low accuracy and lacking sufficient image information, and being unable to truly restore the characteristics of the underwater target.

[0006] To achieve the above objectives, the present invention proposes a method for three-dimensional reconstruction of underwater targets, which is applied to an underwater laser scanning device. The underwater laser scanning device includes three line lasers. The line laser beams emitted by the three line lasers are projected onto the surface of the underwater target and intersect at the same target point. The optical axis of any one of the line lasers is not located in the plane defined by the optical axes of the other two line lasers.

[0007] The underwater target 3D reconstruction method comprises the following steps:

[0008] The laser stripe image reflected by the underwater target is collected by the point cloud camera, and the high-order texture image of the underwater target is collected by the texture camera;

[0009] Performing a point cloud reconstruction operation on the laser stripe image to obtain a laser point cloud grid model;

[0010] Performing a library construction operation on the high-order texture image to obtain a texture feature and point cloud library;

[0011] The features of the laser point cloud mesh model are matched with the texture features and the features of the point cloud library, and the laser point cloud mesh model is texture mapped and colored based on the matching results to generate a three-dimensional reconstructed image of the underwater target with high-order texture information.

[0012] In one embodiment, before the step of performing a point cloud reconstruction operation on the laser stripe image, the step includes:

[0013] Performing a stripe key frame extraction operation on the laser stripe image to obtain an initial stripe key frame image;

[0014] Performing denoising and binarization processing on the initial stripe key frame image to obtain a preprocessed stripe key frame image;

[0015] Performing frame orientation classification container processing on the pre-processed stripe key frame image to obtain three directional laser stripe images, wherein the three directional laser stripe images correspond one-to-one to three linear laser stripes formed by projecting a linear laser beam onto an underwater target surface;

[0016] The step of performing a point cloud reconstruction operation on the laser stripe image comprises:

[0017] The point cloud reconstruction operation is performed on the three directional laser stripe images.

[0018] In one embodiment, the step of performing frame-oriented classification container processing on the pre-processed stripe key frame image includes:

[0019] Three types of mask sub-images are established according to the three independent stripe vector directions corresponding to the three line laser stripes formed by the line laser beam projected onto the underwater target surface;

[0020] The pre-processed stripe key frame image is matrix multiplied with the three types of mask sub-images to obtain three directional laser stripe images.

[0021] In one embodiment, the step of performing the point cloud reconstruction operation on the three directional laser stripe images includes:

[0022] The three directional laser stripe images are sequentially subjected to stripe center extraction operations, point cloud sampling and processing operations, and depth information projection operations to obtain three-dimensional depth point cloud data.

[0023] In one embodiment, the step of performing the point cloud reconstruction operation on the three directional laser stripe images further includes:

[0024] Acquiring spatial movement information of a motion control platform where the underwater laser scanning device is located; the spatial movement information includes a stroke parameter or a posture parameter of the motion control platform;

[0025] Performing coordinate transformation on the three-dimensional depth point cloud data using the coordinate transformation matrix of the spatial movement information to obtain three-dimensional point cloud reconstruction data;

[0026] The three-dimensional point cloud reconstruction data is gridded to obtain the laser point cloud grid model.

[0027] In one embodiment, before the step of performing a library construction operation on the high-order texture image, the method includes:

[0028] Performing an image key frame extraction operation on the high-order texture image to obtain an initial texture key frame image;

[0029] Performing denoising and binarization processing on the initial texture key frame image to obtain a preprocessed texture key frame image;

[0030] Performing frame orientation classification container processing on the pre-processed texture key frame image to obtain three orientation texture images, wherein the three orientation texture images correspond one to one to three line laser stripes formed by projecting a line laser beam onto the underwater target surface;

[0031] The step of performing a library construction operation on the high-order texture image includes:

[0032] The library building operation is performed on the three directional texture images.

[0033] In one embodiment, the step of performing frame-oriented classification container processing on the pre-processed texture key frame image includes:

[0034] Three types of mask sub-images are established according to the three independent stripe vector directions corresponding to the three line laser stripes formed by the line laser beam projected onto the underwater target surface;

[0035] The pre-processed texture key frame image is matrix multiplied with the three types of mask sub-images to obtain the three directional texture images.

[0036] In one embodiment, the step of performing the library construction operation on the three oriented texture images includes:

[0037] A texture feature extraction operation and a texture point cloud sampling operation are performed on the three oriented texture images to obtain the texture features and point cloud library.

[0038] In one embodiment, the step of matching the features of the laser point cloud mesh model with the texture features and the features of the point cloud library, and performing texture mapping and color rendering on the laser point cloud mesh model based on the matching results to generate a three-dimensional reconstructed image of the underwater target with high-order texture information includes:

[0039] Matching the grid features in the laser point cloud grid model with the texture features and the texture features in the point cloud library to obtain spatial synchronization parameters;

[0040] Based on the spatial synchronization parameters, the laser point cloud in the laser point cloud grid model is fused with the texture features and the texture point cloud in the point cloud library to generate a three-dimensional reconstructed image of the underwater target with high-order texture information.

[0041] Correspondingly, the present invention further provides a device for 3D reconstruction of underwater targets, comprising:

[0042] An acquisition module is used to acquire laser stripe images reflected by underwater targets through a point cloud camera, and to acquire high-order texture images of underwater targets through a texture camera;

[0043] A laser stripe image processing module is used to perform a point cloud reconstruction operation on the laser stripe image to obtain a laser point cloud grid model;

[0044] A texture image processing module is used to perform a library construction operation on the high-order texture image to obtain texture features and a point cloud library;

[0045] A fusion module is used to match the features of the laser point cloud mesh model with the texture features and the features of the point cloud library, and to perform texture mapping and color rendering on the laser point cloud mesh model based on the matching results to generate a three-dimensional reconstructed image of the underwater target with high-order texture information.

[0046] As a result, as the image processing technology progresses, the underwater three-dimensional spaceship is transformed into a three-dimensional spaceship, which can accurately measure the depth of the underwater target and the shape of the image, so as to realize the three-dimensional reconstruction of the underwater target and the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0048] Figure 1 A schematic diagram of the operating steps of a method for 3D reconstruction of underwater targets provided by one embodiment of the present invention;

[0049] Figure 2 A schematic diagram of the overall process of a method for 3D reconstruction of underwater targets provided by one embodiment of the present invention;

[0050] Figure 3 A schematic diagram of the laser stripe image processing and point cloud reconstruction process of a method for 3D reconstruction of underwater targets provided by one embodiment of the present invention;

[0051] Figure 4 A schematic diagram of the texture image processing and library construction process of a method for 3D reconstruction of underwater targets provided by one embodiment of the present invention;

[0052] Figure 5A schematic diagram of the fusion process of the laser point cloud mesh model, texture features, and point cloud library in the underwater target 3D reconstruction method provided by one embodiment of the present invention;

[0053] Figure 6 A schematic structural diagram of an underwater laser scanning device corresponding to a method for 3D reconstruction of underwater targets provided in one embodiment of the present invention;

[0054] Figure 7 Schematic diagram of a line laser beam emitted by an underwater laser scanning device of the present invention projected onto an underwater target surface;

[0055] Figure 8 A schematic structural diagram of an underwater target 3D reconstruction device provided by one embodiment of the present invention.

[0056] Description of Figure Numbers:

[0057] 1. Line laser; 2. Point cloud camera; 3. Texture camera.

[0058] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0060] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture. If the specific posture changes, the directional indications will also change accordingly.

[0061] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited to "first" and "second" may explicitly or implicitly include at least one of such features. In addition, if "and / or" or "and / or" appears in the full text, its meaning includes three parallel solutions. Taking "A and / or B" as an example, it includes solution A, solution B, or solutions that satisfy both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0062] Three-dimensional reconstruction of underwater target shapes and stereoscopic spatial reconstruction are urgent technical challenges to address diverse underwater scenarios, including detailed terrain mapping in seabed exploration, feature target identification in underwater search and rescue, equipment shape reconstruction in marine engineering maintenance, and object size measurement in marine ranching. These technologies enable 3D scanning of underwater targets (seabed terrain, search and rescue targets, engineering equipment, aquaculture objects, etc.) in real-world ocean scenarios to obtain depth information. The scanned pose data is then overlaid to convert the information into 3D information. Combined with deep learning and data twin technologies, underwater virtual reality simulation is achieved, providing visual, panoramic, stereoscopic, and real-time intelligent perception technology for the utilization and development of marine resources.

[0063] Currently, underwater target shape 3D reconstruction and stereoscopic space reconstruction technologies primarily include underwater acoustic 3D measurement and underwater optical 3D measurement. Underwater acoustic 3D measurement primarily involves the hardware forms of a 2D imaging sonar vertical array and a 3D imaging sonar. These methods offer the advantages of a longer range, but are limited by the inability to measure at close range and low measurement accuracy. Underwater optical 3D measurement primarily involves the hardware forms of underwater binocular cameras and underwater laser scanners. The underwater range of binocular cameras is limited by water turbidity, and obtaining a smaller depth estimation error requires a larger distance between the imaging centers of the two cameras. Underwater laser scanners offer advantages such as a moderate range, high signal resolution, high measurement accuracy, and a low computational effort for image data, making them suitable for near-field, refined 3D measurement scenarios underwater.

[0064] However, existing underwater laser scanners mainly use single-line lasers or parallel linear array lasers as signal sources. When the underwater target being measured has a stepped structure and the stepped structure has an interface parallel to the direction of the linear laser, the step depth of this stepped structure cannot be measured. In other words, underwater laser scanners using single-line lasers or parallel linear array lasers as signal sources have a measurement blind spot when measuring stepped structures. In addition, existing underwater laser scanners only perform three-dimensional reconstruction of the collected laser point cloud. Due to the above defects, the final generated three-dimensional reconstructed image of the underwater target is not accurate and lacks sufficient image information, making it impossible to truly restore the characteristics of the underwater target.

[0065] In order to solve the above problems, the present invention provides a method for reconstructing underwater targets. Three-line lasers intersecting with each other are used as signal sources. For a stepped structure in any cross-sectional direction, at least one line laser beam emitted by the underwater target 3D reconstruction method can be guaranteed to intersect with the step interface of the stepped structure. In this way, the step depth of the stepped structure can be accurately measured, thereby achieving refined 3D measurement of the complex structure of the underwater target without measuring blank blind spots. Moreover, through the corresponding algorithm process, the collected laser stripe image reflected by the underwater target is fused with the high-order texture image of the underwater target to achieve 3D reconstruction of the underwater target with high-order texture coloring, thereby obtaining a 3D reconstructed image of the underwater target with high precision, containing complete image information, and truly restoring the characteristics of the underwater target.

[0066] See also Figure 1 、 Figure 2 、 Figure 6 and Figure 7 The underwater target three-dimensional reconstruction method provided by the present invention is applied to an underwater laser scanning device (refer to Figure 6 ), the underwater laser scanning device includes three line lasers 1, the line laser beams emitted by the three line lasers 1 are projected onto the underwater target surface and intersect at the same target point, and the optical axis of any line laser 1 is not located in the plane determined by the optical axes of the other two line lasers 1;

[0067] The underwater target 3D reconstruction method comprises the following steps:

[0068] The laser stripe image reflected by the underwater target is collected by the point cloud camera 2, and the high-order texture image of the underwater target is collected by the texture camera 3;

[0069] Perform point cloud reconstruction on the laser stripe image to obtain a laser point cloud mesh model;

[0070] Perform library construction operations on high-order texture images to obtain texture features and point cloud libraries;

[0071] The features of the laser point cloud mesh model are matched with the texture features and the features of the point cloud library, and the laser point cloud mesh model is texture mapped and colored based on the matching results to generate a three-dimensional reconstructed image of the underwater target with high-order texture information.

[0072] In this embodiment, by using three intersecting laser lines as the signal source, when the underwater target being measured has a stepped structure, at least one laser line beam is guaranteed to intersect the step interface of the stepped structure in any cross-sectional direction. This allows accurate measurement of the step depth of the stepped structure, enabling refined three-dimensional measurement of complex underwater target structures without measurement blind spots. Furthermore, through a corresponding algorithmic process, this embodiment effectively fuses the laser stripe image reflected from the underwater target, captured by point cloud camera 2, with the high-order texture image of the underwater target, captured by texture camera 3, to achieve three-dimensional reconstruction of the underwater target with high-order texture coloring. Ultimately, a highly accurate, information-rich, and faithfully restored three-dimensional reconstructed image of the underwater target is obtained.

[0073] In the specific implementation process, the three line lasers 1 adopt a three-dimensional spatial cross layout, and the optical axes of the three line lasers 1 form a spatial structure of "two axes coplanar and the third axis orthogonal"; for example, the optical axis of the first line laser 1 and the optical axis of the second line laser 1 are arranged at 90 degrees in a plane, and the optical axis of the third line laser 1 is perpendicular to the plane. The beneficial effect of this three-line laser layout is that when the underwater target to be measured has a stepped structure, no matter what the direction of the stepped interface is, it can be guaranteed that at least one line laser beam will intersect with the stepped interface; for example, Figure 7 As shown in the figure, when the step interface X of the underwater target is parallel to the direction of the first linear laser beam a1, the second linear laser beam a2 or the third linear laser beam a3 will inevitably intersect with the step interface X. Based on this setting, the problem of step depth measurement blind spots caused by the direction of traditional single-line or parallel linear array lasers being parallel to the step interface can be avoided, and omnidirectional depth information capture of complex underwater step-shaped structures can be achieved.

[0074] Then, the laser stripe image reflected by the underwater target is collected by point cloud camera 2. Point cloud camera 2 is a device that can capture the depth information of the surface of objects in the scene. In this embodiment, it is specifically responsible for receiving the reflected light of the underwater target from the linear laser beam to form an image containing depth information; at the same time, high-order texture images of the underwater target are collected by texture camera 3. Texture camera 3 is used to capture texture feature information such as color, lines, and patterns on the surface of the object. High-order texture images refer to texture images with higher resolution and rich details.

[0075] The collected laser streak image is then reconstructed into a point cloud to create a laser point cloud mesh model. Point cloud reconstruction uses a specific algorithm to convert the depth information in a two-dimensional image into a set of point coordinates in three-dimensional space, while the mesh model connects these discrete points through a grid to form a three-dimensional model.

[0076] Then, a library construction operation is performed on the high-order texture image to obtain a texture feature and point cloud library. The library construction operation includes feature extraction and classification of the high-order texture image, so that it can be selectively used according to subsequent needs.

[0077] Finally, the features of the laser point cloud mesh model are matched with the texture features and the features of the point cloud library, and based on the matching results, the laser point cloud mesh model is texture mapped and colored to generate a three-dimensional reconstructed image of the underwater target with high-order texture information; among them, texture mapping is to accurately map the corresponding texture image to the corresponding position of the three-dimensional model, and color rendering is to fill the model with color through computer graphics technology to make it present a realistic visual effect.

[0078] The technical solution based on this embodiment can not only effectively solve the measurement blind spot problem in the existing technology and improve the accuracy of three-dimensional reconstruction of underwater targets, but also enrich the information of the reconstructed image so that it can more realistically restore the characteristics of the underwater target. It has important application value in the fields of marine resource development, underwater search and rescue, marine engineering maintenance, etc.

[0079] In one embodiment, reference Figure 3 , before the step of performing point cloud reconstruction on the laser stripe image, including:

[0080] Performing a stripe key frame extraction operation on the laser stripe image to obtain an initial stripe key frame image;

[0081] Perform denoising and binarization processing on the initial stripe key frame image to obtain a preprocessed stripe key frame image;

[0082] Performing frame-oriented classification container processing on the pre-processed stripe key frame image to obtain three directional laser stripe images, which correspond one-to-one to the three linear laser stripes formed by the linear laser beam projected onto the underwater target surface;

[0083] The steps of performing a point cloud reconstruction operation on the laser stripe image include:

[0084] Perform point cloud reconstruction on three directional laser stripe images.

[0085] In this embodiment, the laser stripe image needs to be preprocessed before performing a point cloud reconstruction operation on the laser stripe image. First, a stripe keyframe extraction operation is performed to obtain an initial stripe keyframe image. Stripe keyframe extraction is the process of selecting a keyframe image containing valid laser stripe information from a continuously acquired image sequence for subsequent processing. The initial stripe keyframe image is then subjected to denoising and binarization processing to obtain a preprocessed stripe keyframe image. Denoising is performed to remove noise interference in the image, and binarization is the process of quantizing the image grayscale values ​​to 0 and 1 to highlight the stripe features. The preprocessed stripe keyframe image is then subjected to frame orientation classification container processing to obtain three directional laser stripe images. Frame orientation classification container processing is the process of classifying the image according to the directional features of the stripes to extract the stripe images corresponding to the line laser beams emitted by the three line lasers 1. When the laser stripe image is subsequently subjected to point cloud reconstruction, the three directional laser stripe images are processed separately to obtain more accurate point cloud data.

[0086] In one embodiment, reference Figure 3 The step of performing frame-oriented classification container processing on the pre-processed stripe key frame image includes:

[0087] Three types of mask sub-images are established according to the three independent stripe vector directions corresponding to the three line laser stripes formed by the line laser beam projected onto the underwater target surface;

[0088] The preprocessed stripe key frame image is matrix multiplied with the three types of mask sub-images to obtain three directional laser stripe images.

[0089] Specifically, a mask sub-image is a template image used to extract stripes in a specific direction. Its creation process is as follows: First, the directional characteristics of the three laser stripes are analyzed to determine the vector direction of each stripe. Then, a corresponding mask sub-image is designed based on each vector direction. The mask sub-image has the same size as the preprocessed stripe keyframe image and has a specific pixel value distribution in the corresponding direction to highlight the stripe characteristics in that direction. Next, the preprocessed stripe keyframe image is matrix-multiplied with the three types of mask sub-images. The specific operation of the matrix multiplication is to multiply each pixel value in the preprocessed stripe keyframe image with the pixel value in the corresponding mask sub-image, resulting in three new images, namely, three directional laser stripe images. In each directional laser stripe image, only the stripe information in the corresponding direction is retained, and the stripe information in other directions is suppressed or removed.

[0090] In this way, the three laser stripes in the original image can be extracted separately to provide a clear stripe image for the subsequent point cloud reconstruction operation.

[0091] In one embodiment, reference Figure 3, the steps of performing a point cloud reconstruction operation on three directional laser stripe images include:

[0092] The three directional laser stripe images are sequentially subjected to stripe center extraction, point cloud sampling and processing, and depth information projection operations to obtain three-dimensional depth point cloud data.

[0093] Specifically, the fringe center extraction operation determines the precise location of the fringe within the image. Sub-pixel edge detection algorithms (such as the Sobel operator and the Canny operator) are typically used, combined with fringe morphological features (such as fringe width and contrast) to accurately extract the fringe center pixel coordinates. Point cloud sampling and processing are then performed. Point cloud sampling converts the extracted fringe center pixel coordinates into three-dimensional point cloud data. Based on the intrinsic and extrinsic parameter matrices of point cloud camera 2 and the geometric relationship between the laser stripes, a projection model is used to convert the two-dimensional image coordinates into three-dimensional space coordinates. Point cloud processing includes operations such as noise removal and filtering and smoothing to improve the quality of the point cloud data. Next, depth information projection is performed. Depth information projection projects the processed point cloud data into a world coordinate system. Combined with the scanning device's position information, the precise location of each point cloud data point in the world coordinate system is determined, resulting in three-dimensional depth point cloud data. This three-dimensional depth point cloud data contains depth information of the underwater target surface, providing the basis for subsequent gridding and three-dimensional reconstruction.

[0094] In one embodiment, reference Figure 3 The step of performing a point cloud reconstruction operation on the three directional laser stripe images further includes:

[0095] Acquiring spatial movement information of the motion control platform where the underwater laser scanning device is located; the spatial movement information includes travel parameters or posture parameters of the motion control platform;

[0096] The coordinate transformation matrix of the spatial movement information is used to transform the three-dimensional depth point cloud data to obtain three-dimensional point cloud reconstruction data;

[0097] The 3D point cloud reconstruction data is meshed to obtain a laser point cloud mesh model.

[0098] Specifically, the motion control platform includes an underwater fixed platform and an underwater mobile platform. The underwater fixed platform is typically equipped with a stepper motor, which can drive the underwater laser scanning device for precise movement in the lateral, longitudinal, and rotational directions. The stepper motor receives control signals to drive the underwater laser scanning device to move according to a set step length and direction. Its travel parameters (such as movement distance and movement speed) can be fed back to the control module in real time. The underwater mobile platform can be an AUV (Autonomous Underwater Vehicle) or ROV (Remote Operated Vehicle). The underwater mobile platform carries its own power system and navigation system and can move autonomously along a preset path or be remotely controlled by an operator, thereby driving the underwater laser scanning device mounted on the underwater mobile platform to move according to a preset pattern. The underwater mobile platform's posture parameters (such as position coordinates and attitude angle) can be fed back to the control module in real time. The spatial movement information of the motion control platform reflects the position and attitude changes of the underwater laser scanning device during image data acquisition and serves as the basis for subsequent coordinate transformation.

[0099] After obtaining the spatial movement information, the coordinate transformation matrix of the spatial movement information is used to transform the coordinates of the three-dimensional depth point cloud data obtained after the depth information projection operation to obtain three-dimensional point cloud reconstruction data. Among them, the coordinate transformation matrix is ​​calculated based on the spatial movement information and is used to transform the point cloud data from the local coordinate system of the laser scanning device to the global coordinate system to ensure that all data are in the same reference system, so as to achieve accurate splicing and fusion of the data in the subsequent process. Specifically, the coordinate transformation matrix may include a translation vector and a rotation matrix. The translation vector reflects the position change of the motion control platform, and the rotation matrix reflects the posture change of the motion control platform. The coordinates of each point cloud data point are transformed through matrix operations so that it can correspond to the accurate position in the global coordinate system. The three-dimensional point cloud reconstruction data obtained after the above-mentioned coordinate transformation can truly reflect the distribution of underwater targets in space and provide accurate basic data for further gridding processing and three-dimensional reconstruction.

[0100] Finally, the 3D point cloud reconstruction data is meshed to produce a laser point cloud mesh model. Meshing can utilize algorithms such as Delaunay triangulation and Poisson reconstruction to connect the discrete point cloud data into a continuous mesh surface, thus forming a complete 3D model. These algorithms analyze the topological relationships and geometric features of the point cloud data to generate a mesh that conforms to the actual surface shape of the object, thereby achieving refined 3D reconstruction of underwater targets.

[0101] In one embodiment, reference Figure 4 , before the step of library building operation for high-order texture images, including:

[0102] Performing an image key frame extraction operation on the high-order texture image to obtain an initial texture key frame image;

[0103] Performing denoising and binarization processing on the initial texture key frame image to obtain a preprocessed texture key frame image;

[0104] The pre-processed texture key frame image is processed by frame orientation classification container to obtain three orientation texture images. The three orientation texture images correspond one to one to three line laser stripes formed by the line laser beam projected onto the underwater target surface.

[0105] The steps for building a library for high-order texture images include:

[0106] Perform a library build operation on three directional texture images.

[0107] In this embodiment, before performing a library construction operation on the high-order texture image, a series of pre-processing operations need to be performed on the high-order texture image. The specific steps are as follows:

[0108] First, an image keyframe extraction operation is performed on the high-order texture image to obtain the initial texture keyframe image. Image keyframe extraction involves screening keyframes containing valid texture information from a continuously acquired high-order texture image sequence. Keyframe selection is typically based on factors such as the degree of difference between frames, the richness of texture information, and representativeness of underwater target features. By analyzing the similarities and differences between frames in the image sequence, keyframes that best represent the changes in underwater target texture features are selected and used as the basis for subsequent processing.

[0109] The initial texture keyframe image is then subjected to denoising and binarization to produce a preprocessed texture keyframe image. Denoising aims to remove image clutter caused by factors such as underwater environmental interference and equipment noise, thereby improving image purity. Common denoising algorithms include Gaussian filtering and median filtering. Binarization quantizes the pixel values ​​in the image to two extremes, 0 and 1. By setting an appropriate threshold, texture features are separated from the background, enhancing texture contrast and recognizability, facilitating subsequent feature extraction and processing.

[0110] The preprocessed texture keyframe image is then subjected to frame-oriented classification container processing to obtain three oriented texture images. This process is similar to the frame-oriented classification container processing of laser streak images. Similarly, three types of mask sub-images are established based on the three independent streak vector directions corresponding to the three linear laser streaks formed by the linear laser beam projected onto the underwater target surface. Matrix multiplication is then performed between the preprocessed texture keyframe image and these three types of mask sub-images. Through the screening effect of the mask sub-images, oriented texture images corresponding to the three linear laser streaks are extracted. Each oriented texture image corresponds to the texture features of a linear laser streak region, thus decomposing the complex texture image into three texture sub-images with clear directional features, providing clear and oriented texture data for subsequent texture feature extraction and library construction.

[0111] In one embodiment, reference Figure 4 The step of performing frame-oriented classification container processing on the pre-processed texture key frame image comprises:

[0112] Three types of mask sub-images are established according to the three independent stripe vector directions corresponding to the three line laser stripes formed by the line laser beam projected onto the underwater target surface;

[0113] The preprocessed texture keyframe image is matrix multiplied with the three types of mask sub-images to obtain three oriented texture images.

[0114] Specifically, a mask sub-image is a template image used to extract texture in a specific direction. Its creation process is as follows: First, the directional characteristics of the three laser stripes are analyzed to determine the vector direction of each stripe. Then, a corresponding mask sub-image is designed based on each vector direction. The mask sub-image has the same size as the pre-processed texture keyframe image and has a specific pixel value distribution in the corresponding direction to highlight the stripe characteristics in the corresponding direction. Next, the pre-processed texture keyframe image is matrix-multiplied with the three types of mask sub-images. The specific operation of the matrix multiplication is to multiply each pixel value in the pre-processed texture keyframe image with the pixel value in the corresponding mask sub-image, resulting in three new images, namely three directional texture images. In each directional texture image, only the texture information in the corresponding direction is retained, and the texture information in other directions is suppressed or removed, thus providing a clear texture image for subsequent library construction operations.

[0115] In one embodiment, reference Figure 4 , the steps of performing library construction operations for three directional texture images include:

[0116] The texture feature extraction operation and texture point cloud sampling operation are performed on the three oriented texture images to obtain the texture features and point cloud library.

[0117] Specifically, the texture feature extraction operation uses algorithms such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Robust Features), and ORB (Binary Rotation Invariant Features) to identify and extract key feature points, edges, corners, and other information from oriented texture images. These feature information can reflect the texture details and structural features of the underwater target surface, and are an important basis for subsequent texture matching and mapping. The texture point cloud sampling operation associates the extracted texture features with the corresponding three-dimensional point cloud data, and generates a texture feature and point cloud library by establishing a correspondence between texture feature points and three-dimensional space points. This texture feature and point cloud library not only contains the descriptive information of the texture features, but also records the position of each texture feature in three-dimensional space, providing rich data support for subsequent texture mapping and three-dimensional reconstruction.

[0118] In one embodiment, reference Figure 5 The steps of matching the features of the laser point cloud mesh model with the texture features and the features of the point cloud library, and performing texture mapping and color rendering on the laser point cloud mesh model based on the matching results to generate a three-dimensional reconstructed image of the underwater target with high-order texture information include:

[0119] Match the mesh features in the laser point cloud mesh model with the texture features and the texture features in the point cloud library to obtain spatial synchronization parameters;

[0120] Based on the spatial synchronization parameters, the laser point cloud in the laser point cloud meshing model is fused with the texture features and the texture point cloud in the point cloud library to generate a three-dimensional reconstructed image of the underwater target with high-order texture information.

[0121] In this embodiment, the matching process compares the mesh features in the laser point cloud mesh model with the texture features and texture features in the point cloud library. Mesh features include geometric information such as the coordinates, normal vectors, and curvature of the mesh vertices, while texture features include the color histogram and texture feature vectors (such as contrast and correlation calculated using the GLCM gray-level co-occurrence matrix) in the texture image, as well as key point descriptors (such as SIFT and SURF feature descriptors). The matching algorithm typically uses the nearest neighbor method, the iterative closest point algorithm (ICP), or a feature-based matching algorithm. For example, key point descriptors can be used to find similar feature points between the laser point cloud mesh model and the texture features and point cloud library, and by constructing a correspondence between the feature points, a preliminary matching of the mesh model and the texture features can be achieved; further, the matching results can be optimized in combination with geometric constraints (such as distance constraints, angle constraints, etc.), erroneous matching point pairs can be eliminated, and matching point pairs with high reliability can be retained, thereby obtaining accurate spatial synchronization parameters; spatial synchronization parameters include rotation matrices, translation vectors, and scaling factors, which are used to describe the spatial relationship between the laser point cloud mesh model and the texture features and point cloud library.

[0122] Based on the spatial synchronization parameters, the specific process of fusing the laser point cloud in the laser point cloud mesh model with the texture features and the texture point cloud in the point cloud library is as follows: a unified coordinate system is established according to the spatial synchronization parameters, the laser point cloud mesh model is converted to the coordinate system, and the texture features and the texture point cloud in the point cloud library are subjected to corresponding coordinate transformations so that they are located in the same spatial reference system as the laser point cloud mesh model; then, the texture features can be mapped to the corresponding positions of the laser point cloud mesh model through interpolation algorithms (such as bilinear interpolation, nearest neighbor interpolation, etc.); for each vertex or mesh patch in the mesh model, the texture features and the point cloud library are found according to their positions in the unified coordinate system. To the nearest texture pixel point or obtain the corresponding texture color value through interpolation, and assign it to the corresponding position of the model; in addition, the lighting model and material properties must be considered, and the texture color must be adjusted and rendered to simulate the real lighting effect and material reflection characteristics, so as to generate a three-dimensional reconstructed image of the underwater target with high-order texture information; for example, the Phong lighting model can be used, combined with the parameter settings of texture color, ambient light, diffuse light and specular light, to calculate the final color value of each vertex or pixel, so that the reconstructed image presents a more realistic and delicate visual effect, enhances its three-dimensional sense and realism, and provides a high-quality visualization model for subsequent underwater target recognition, analysis and application.

[0123] Through the deep integration of laser point cloud mesh model, texture features and point cloud library, the three-dimensional reconstructed image of underwater targets can be unified in geometric shape and texture details, which can significantly improve the quality and information integrity of the reconstructed image.

[0124] Correspondingly, see Figure 8 The embodiment of the present invention further provides a device for 3D reconstruction of underwater targets, the device comprising:

[0125] An acquisition module 10 is configured to acquire laser stripe images reflected by underwater targets through a point cloud camera 2 and to acquire high-order texture images of underwater targets through a texture camera 3;

[0126] The laser streak image processing module 20 is used to perform a point cloud reconstruction operation on the laser streak image to obtain a laser point cloud grid model;

[0127] The texture image processing module 30 is used to perform a library construction operation on the high-order texture image to obtain texture features and a point cloud library;

[0128] The fusion module 40 is used to match the features of the laser point cloud mesh model with the texture features and the features of the point cloud library, and to perform texture mapping and color rendering on the laser point cloud mesh model based on the matching results to generate a three-dimensional reconstructed image of the underwater target with high-order texture information.

[0129] It should be noted that other contents of the underwater target three-dimensional reconstruction method and device disclosed in the present invention can be found in the prior art and will not be described in detail here.

[0130] The above description is merely an exemplary embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformation made by utilizing the contents of the present invention's description and drawings under the technical concept of the present invention, or directly / indirectly applied in other related technical fields, is included in the patent protection scope of the present invention.

Claims

1. A method for 3D reconstruction of underwater targets, applied to an underwater laser scanning device, characterized in that: The underwater laser scanning device includes three line lasers, the line laser beams emitted by the three line lasers are projected onto the underwater target surface and intersect at the same target point, and the optical axis of any one of the line lasers is not located in the plane defined by the optical axes of the other two line lasers; The underwater target 3D reconstruction method comprises the following steps: The laser stripe image reflected by the underwater target is collected by the point cloud camera, and the high-order texture image of the underwater target is collected by the texture camera; Performing a point cloud reconstruction operation on the laser stripe image to obtain a laser point cloud grid model; Performing a library construction operation on the high-order texture image to obtain a texture feature and point cloud library; The features of the laser point cloud mesh model are matched with the texture features and the features of the point cloud library, and the laser point cloud mesh model is texture mapped and colored based on the matching results to generate a three-dimensional reconstructed image of the underwater target with high-order texture information.

2. The underwater target 3D reconstruction method according to claim 1, wherein: Before the step of performing a point cloud reconstruction operation on the laser stripe image, the method includes: Performing a stripe key frame extraction operation on the laser stripe image to obtain an initial stripe key frame image; Performing denoising and binarization processing on the initial stripe key frame image to obtain a preprocessed stripe key frame image; Performing frame orientation classification container processing on the pre-processed stripe key frame image to obtain three directional laser stripe images, wherein the three directional laser stripe images correspond one-to-one to three linear laser stripes formed by projecting a linear laser beam onto an underwater target surface; The step of performing a point cloud reconstruction operation on the laser stripe image comprises: The point cloud reconstruction operation is performed on the three directional laser stripe images.

3. The underwater target 3D reconstruction method according to claim 2, wherein: The step of performing frame-oriented classification container processing on the pre-processed stripe key frame image comprises: Three types of mask sub-images are established according to the three independent stripe vector directions corresponding to the three line laser stripes formed by the line laser beam projected onto the underwater target surface; The pre-processed stripe key frame image is matrix multiplied with the three types of mask sub-images to obtain three directional laser stripe images.

4. The method for 3D reconstruction of underwater targets according to claim 2, wherein: The step of performing the point cloud reconstruction operation on the three directional laser stripe images includes: The three directional laser stripe images are sequentially subjected to stripe center extraction operations, point cloud sampling and processing operations, and depth information projection operations to obtain three-dimensional depth point cloud data.

5. The method for 3D reconstruction of underwater targets according to claim 4, wherein: The step of performing the point cloud reconstruction operation on the three directional laser stripe images further includes: Acquiring spatial movement information of a motion control platform where the underwater laser scanning device is located; the spatial movement information includes a stroke parameter or a posture parameter of the motion control platform; Performing coordinate transformation on the three-dimensional depth point cloud data using the coordinate transformation matrix of the spatial movement information to obtain three-dimensional point cloud reconstruction data; The three-dimensional point cloud reconstruction data is gridded to obtain the laser point cloud grid model.

6. The method for 3D reconstruction of underwater targets according to claim 1, wherein: Before the step of performing a library construction operation on the high-order texture image, the method includes: Performing an image key frame extraction operation on the high-order texture image to obtain an initial texture key frame image; Performing denoising and binarization processing on the initial texture key frame image to obtain a preprocessed texture key frame image; Performing frame orientation classification container processing on the pre-processed texture key frame image to obtain three orientation texture images, wherein the three orientation texture images correspond one to one to three line laser stripes formed by projecting a line laser beam onto the underwater target surface; The step of performing a library construction operation on the high-order texture image includes: The library building operation is performed on the three directional texture images.

7. The method for 3D reconstruction of underwater targets according to claim 6, wherein: The step of performing frame-oriented classification container processing on the pre-processed texture key frame image comprises: Three types of mask sub-images are established according to the three independent stripe vector directions corresponding to the three line laser stripes formed by the line laser beam projected onto the underwater target surface; The pre-processed texture key frame image is matrix multiplied with the three types of mask sub-images to obtain the three directional texture images.

8. The method for 3D reconstruction of underwater targets according to claim 6, wherein: The step of performing the library construction operation on the three directional texture images includes: A texture feature extraction operation and a texture point cloud sampling operation are performed on the three oriented texture images to obtain the texture features and point cloud library.

9. The method for three-dimensional reconstruction of an underwater target according to any one of claims 1 to 8, characterized in that: The step of matching the features of the laser point cloud mesh model with the texture features and the features of the point cloud library, and performing texture mapping and color rendering on the laser point cloud mesh model based on the matching results to generate a three-dimensional reconstructed image of the underwater target with high-order texture information includes: Matching the grid features in the laser point cloud grid model with the texture features and the texture features in the point cloud library to obtain spatial synchronization parameters; Based on the spatial synchronization parameters, the laser point cloud in the laser point cloud grid model is fused with the texture features and the texture point cloud in the point cloud library to generate a three-dimensional reconstructed image of the underwater target with high-order texture information.

10. A device for three-dimensional reconstruction of underwater targets, characterized in that: The underwater target three-dimensional reconstruction device comprises: An acquisition module is used to acquire laser stripe images reflected by underwater targets through a point cloud camera, and to acquire high-order texture images of underwater targets through a texture camera; A laser stripe image processing module is used to perform a point cloud reconstruction operation on the laser stripe image to obtain a laser point cloud grid model; A texture image processing module is used to perform a library construction operation on the high-order texture image to obtain texture features and a point cloud library; A fusion module is used to match the features of the laser point cloud mesh model with the texture features and the features of the point cloud library, and to perform texture mapping and color rendering on the laser point cloud mesh model based on the matching results to generate a three-dimensional reconstructed image of the underwater target with high-order texture information.

Images