3D modeling method and device of underwater target and underwater robot

Abstract

The application relates to the technical field of underwater detection, and discloses a 3D modeling method and device for an underwater target and an underwater robot. The method comprises the following steps: performing fixed-distance observation on a to-be-detected object according to a preset path, and acquiring a two-dimensional observation image in real time; performing real-time identification on the image, and confirming a target area; for the identified target area, based on two-dimensional visual data of the target area, a multi-view three-dimensional reconstruction algorithm is used to construct a local three-dimensional model, and corresponding navigation pose data of the underwater robot is recorded synchronously; after the observation is completed, based on each local three-dimensional model and the corresponding navigation pose data, an overall three-dimensional reference model of the to-be-detected object is constructed; finally, according to the navigation pose data, each local three-dimensional model is filled into the three-dimensional reference model, and a complete three-dimensional model is generated. The application combines selective local fine modeling and overall reference model construction based on the pose, greatly reduces the amount of image data required to be processed, and significantly improves the modeling efficiency.

Description

Technical Field

[0001] This invention relates to the field of underwater detection technology, and in particular to a 3D modeling method, device and underwater robot for underwater targets. Background Technology

[0002] Underwater structures, especially critical load-bearing components like bridge piers, are subjected to complex environments such as water erosion, corrosion, and biofouling over long periods. Their structural integrity directly impacts the safety and durability of the entire project. Therefore, regular safety monitoring and defect diagnosis of underwater structures are of paramount importance. With the advancement of underwater robotics technology, utilizing underwater robots equipped with optical cameras for observation and exploration has become a primary technical means of obtaining information about the appearance of underwater structures.

[0003] Currently, a common implementation method involves an underwater robot circling the target structure to systematically collect continuous image or video data of its surface. Subsequently, based on this image data, computer vision technology is used to perform 3D reconstruction, thereby generating a visualized 3D model of the underwater structure to assist engineers in analysis and evaluation.

[0004] However, to fully cover the structural surface and avoid omissions, massive amounts of image data are usually required. Performing 3D reconstruction on all the data involves enormous computational demands, processing time, and storage costs, making it difficult to meet the urgent needs of engineering sites for timely inspection results. Summary of the Invention

[0005] Based on this, it is necessary to address the technical problem of large data volume in the existing technology for 3D modeling of underwater targets, and propose a 3D modeling method, device and underwater robot for underwater targets.

[0006] Firstly, a 3D modeling method for underwater targets is provided, the method comprising: The underwater robot is controlled to perform fixed-distance observation of the object under test according to a preset navigation path, and two-dimensional observation images are acquired in real time through a camera device during the observation process. The two-dimensional observation image is identified in real time to confirm the existence of the target area; For the identified target region, based on the two-dimensional visual data of the target region, a multi-view approach is adopted. Figure 3 The 3D reconstruction algorithm constructs a local 3D model of the target area, and simultaneously records the navigation pose data of the underwater robot when constructing each local 3D model; After completing the observation of the object under test, a three-dimensional reference model of the object under test is constructed based on each of the local three-dimensional models and the navigation pose data corresponding to each of the local three-dimensional models. Based on the navigation pose data corresponding to each of the local 3D models, each of the local 3D models is filled into the 3D reference model to generate a 3D model of the object under test.

[0007] Secondly, a 3D modeling device for underwater targets is provided, the device comprising: The acquisition module is used to control the underwater robot to perform fixed-distance observation of the object to be measured according to a preset navigation path. During the observation process, two-dimensional observation images are acquired in real time through a camera device. The identification module is used to identify the two-dimensional observation image in real time to confirm the existence of the target area; The construction module is used to, based on the two-dimensional visual data of the identified target region, employ multi-view... Figure 3 The 3D reconstruction algorithm constructs a local 3D model of the target area, and simultaneously records the navigation pose data of the underwater robot when constructing each local 3D model; The construction module is further configured to, after completing the observation of the object under test, construct a three-dimensional reference model of the object under test based on each of the local three-dimensional models and the navigation pose data corresponding to each of the local three-dimensional models; The generation module is used to fill the three-dimensional reference model with the navigation pose data corresponding to each of the local three-dimensional models to generate a three-dimensional model of the object to be tested.

[0008] Thirdly, an underwater robot is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned 3D modeling method for underwater targets.

[0009] Beneficial effects: By introducing a real-time target area identification and selective local fine-grained modeling mechanism, the data processing scope is focused from the entire structural surface to key defects or feature areas, thereby significantly reducing the total amount of image data required for processing. Furthermore, the strategy of using navigation pose data to associate local models and construct an overall reference model replaces the traditional computational process that relies on massive amounts of image data for overall reconstruction. Therefore, this application significantly reduces the demands on computing resources, storage space, and processing time in the 3D reconstruction process, effectively overcoming the efficiency bottleneck caused by processing full-volume data in existing technologies, and meeting the stringent timeliness requirements of underwater engineering inspection. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] in: Figure 1 This is a flowchart of a 3D modeling method for an underwater target in one embodiment; Figure 2 A schematic diagram illustrating the construction process of a local 3D model provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the process of constructing a local three-dimensional model based on a second reference dimension, provided for an embodiment of this application; Figure 4 Another schematic diagram illustrating the process of constructing a local three-dimensional model based on a second reference dimension, as provided in an embodiment of this application; Figure 5 A schematic diagram illustrating the process of generating a three-dimensional reference model provided in an embodiment of this application; Figure 6 This is another schematic diagram illustrating the process of generating a three-dimensional reference model provided in an embodiment of this application; Figure 7 This is a schematic diagram of the projection of a local 3D model onto a reference plane; Figure 8 This is a schematic diagram of filling each local three-dimensional model into a reference three-dimensional model, provided in an embodiment of this application; Figure 9 This is a structural block diagram of a 3D modeling device for an underwater target in one embodiment; Figure 10 This is a structural block diagram of an underwater robot in one embodiment. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] The underwater observation system of this application includes: a management device, a surface base station, and at least one underwater robot. The management device can communicate with the underwater robot via cable or the surface base station. In some embodiments, the management device can also be located on the surface base station. The underwater robot mentioned above includes, but is not limited to, types of underwater robots such as remotely operated vehicles (ROVs) and autonomous remotely controlled vehicles (ARVs), and can also be underwater detection equipment, underwater submarine equipment, or other underwater operation equipment; this application does not impose any limitations on this.

[0014] The management equipment is installed in an aquatic environment to plan navigation paths for underwater robots to observe underwater objects (such as bridge piers, ship hulls, etc.), and to configure the navigation paths to designated underwater robots. The underwater robots then observe the objects based on the navigation paths to monitor for defects such as cracks and corrosion. The management equipment can be a mobile device, tablet computer, or fixed computer, etc., and this application does not impose any restrictions on this.

[0015] Floating base stations can also take the form of ship hulls or other waterborne equipment. They are typically equipped with GNSS (Global Navigation Satellite System) and a USBL (Ultra-Short Baseline) transducer array positioned below the water surface. Floating base stations also serve as communication hubs between management equipment and underwater robots, handling task scheduling and ensuring coordinated responses from management equipment commands and underwater robots. In some embodiments, the floating base station also has the capability to supply power to the underwater robot.

[0016] In this embodiment, the underwater robot is equipped with a USBL transponder, which works in conjunction with the USBL transducer array set up by the surface base station for cooperative positioning. The positioning principle is as follows: the surface base station obtains its own position coordinates based on the GNSS module, the underwater robot uses the USBL transponder to measure the relative position offset between itself and the surface base station, and then calculates its own current position coordinates based on the relative position offset and the position coordinates of the surface base station.

[0017] The present invention will now be described in detail through specific embodiments.

[0018] Please see Figure 1 As shown, Figure 1 A flowchart illustrating a 3D modeling method for underwater targets provided in an embodiment of the present invention includes the following steps: S1. Control the underwater robot to perform fixed-distance observation of the object under test according to the preset navigation path, and acquire two-dimensional observation images in real time through the camera device during the observation process.

[0019] Before the mission begins, a pre-defined navigation path must be planned in the mission planning software based on the known prior dimensions, shape, and geographical location of the object under test. This path is typically composed of a series of ordered three-dimensional waypoints, and its core design ensures that the underwater robot can completely sweep across all the surfaces of the object under test with a stable attitude and constant relative distance. Distance-based observation refers to the real-time (e.g., latency less than 100ms) feedback of the distance between the robot and the object surface via an active ranging sensor throughout the observation journey. The robot's attitude and thruster output are dynamically adjusted by the control system to maintain this distance within a preset threshold range. While navigating along the pre-defined path and maintaining a constant distance, the mission planning module automatically triggers the optical camera mounted on the robot's gimbal to periodically capture or continuously record video based on the navigation speed or time interval. Each frame of the acquired two-dimensional observation image is timestamped synchronously with the high-precision combined navigation data at the time of acquisition for subsequent processing.

[0020] For example, when performing visual inspection on a subsea water pipeline with a known length of 50 meters and a diameter of 3 meters, the operator can design a "bow" or spiral 3D navigation path parallel to the pipeline and two meters away from its outer surface, using the pipeline's 3D model or centerline as a reference in the task planning interface. After the underwater robot is launched, its forward-looking sonar or laser rangefinder continuously measures the actual distance to the pipeline wall. The navigation control adjusts the robot's lateral position in real time by comparing the measured value with the preset two-meter distance, thus maintaining a constant observation distance. During the robot's autonomous navigation along this path, its onboard corrosion-resistant high-definition camera, with a gimbal-stabilized attitude, triggers shooting based on the navigation speed and time intervals, simultaneously recording pose data. This results in a sequence of clear images covering the entire outer surface of the pipeline with sufficient overlap. Each image's storage information accurately records the underwater robot's longitude, latitude, depth, roll angle, pitch angle, and heading angle at the time of shooting. The latitude and longitude are obtained through GNSS and USBL collaborative positioning inversion from the surface base station.

[0021] Furthermore, methods for underwater robots to perform ranged observations include: The instantaneous distance between the device and the surface of the object being measured is measured in real time using a Doppler log. The difference between the instantaneous distance and the preset observation distance threshold is calculated, and the lateral position of the underwater robot is adjusted based on the difference to stabilize the instantaneous distance within the fluctuation range allowed by the observation distance threshold.

[0022] Specifically, a Doppler odometer is mounted at the front of the robot, and its acoustic transducer array emits a high-frequency sound beam forward and downward at a certain angle. When the sound beam reaches the surface of the object being measured, some of the sound energy is reflected back to the transducer. By accurately measuring the time difference between sound wave emission and reception, i.e., the time of flight, and based on the known speed of sound in water, the straight-line slant distance between the front of the robot and the reflection point on the object surface can be directly calculated. To achieve lateral distance control perpendicular to the robot's heading, this slant distance needs to be converted into the normal distance or horizontal lateral distance between the robot and the object surface through geometric relationships. This requires coordinate transformation calculations combining the fixed pitch angle of the odometer beam and the roll and pitch angle data provided in real time by the robot's attitude sensors. This measurement process is performed continuously at a frequency of tens of hertz, thereby outputting a series of continuous, real-time instantaneous distance data streams, providing core feedback for subsequent closed-loop control.

[0023] For example, an underwater robot is equipped with a Doppler logger operating at a frequency of 1.2 MHz at its front, with a fixed pitch angle of 15 degrees downwards for its acoustic beam. When the robot travels to a position approximately three meters from the seabed pipeline, the logger emits an acoustic pulse and receives the echo, measuring a slant distance of 3.1 meters. Simultaneously, the robot's inertial measurement unit measures a slight pitch of the robot body, with an attitude of -2 degrees pitch. Based on the beam pitch angle, the robot's pitch angle, and the measured 3.1-meter slant distance, trigonometric calculations are used to determine the actual normal distance between the robot and the pipeline surface as 2.98 meters, which is taken as the instantaneous distance at the current moment.

[0024] The underwater robot continuously receives instantaneous distance measurements and compares them with a preset observation distance threshold to calculate a real-time error value, or difference. This observation distance threshold is an ideal working distance preset based on factors such as imaging quality, obstacle avoidance, and observation efficiency. The control system employs an appropriate control algorithm, such as proportional-integral-derivative (PID) control, to process this error value and generate corresponding control commands. The core of these control commands is adjusting the thrust output of the underwater robot's lateral thrusters or vector thrusters. Specifically, when the instantaneous distance exceeds the threshold, it means the robot is too far from the object, and the control command instructs the robot to move laterally closer to the object; conversely, it instructs the robot to move laterally away from the object. Through this continuous feedback and adjustment, distance deviations caused by water flow disturbances, robot inertia, or changes in path curvature can be actively offset, thereby dynamically stabilizing the actual working distance between the robot and the surface of the object under test within a permissible small fluctuation range near the preset threshold, providing stable and consistent observation geometry for subsequent image acquisition.

[0025] S2. Real-time identification of two-dimensional observation images to confirm the existence of the target area.

[0026] The process begins with preprocessing the raw 2D observation image acquired in step S1 to overcome inherent limitations of underwater imaging. Preprocessing typically includes color correction based on physical models or deep learning to restore true color tones, and filtering algorithms to suppress scattering noise caused by suspended particles. The preprocessed image is then input into a pre-trained deep learning target detection model for inference. This model is specifically trained to identify common surface defects in underwater structures, such as cracks and corrosion. The model analyzes the entire image and outputs the bounding box coordinates, category labels, and a confidence score representing the reliability of one or more potential defect regions. To confirm the existence of a target region, a confidence threshold is set. Only when the confidence score of a detection result exceeds this threshold is the defect considered to exist in the current image, and the corresponding bounding box region is formally marked as a candidate target region for further processing. The entire processing flow requires algorithmic optimization and hardware acceleration to ensure that the processing latency of a single frame meets the real-time requirements of continuous observation by the underwater robot.

[0027] For example, when the robot acquires an image of the hull of a sunken ship, it first calls the pre-loaded underwater image enhancement model to restore the color and contrast of the bluish-green image. Next, the enhanced image is fed into a pre-loaded YOLOv5 model trained on a large number of underwater steel defect images. This model undergoes pruning and quantization to adapt to the computing power of the embedded platform. After inference, the model outputs a predicted bounding box indicating the presence of a "crack" in the image, with a confidence level of 0.85. The preset confidence threshold is 0.7; since 0.85 is higher than this threshold, the presence of a crack target area in the image is confirmed, and the coordinates of the predicted bounding box are recorded.

[0028] It should be noted that after completing the real-time identification in step S2, if it is determined that there is no target area in the observed image that meets the feature threshold condition, the method will skip steps S3 to S5, that is, it will not perform the construction of local three-dimensional model, construction and fusion of three-dimensional reference model for the target area, and will continue to perform fixed-distance observation along the preset navigation path or end the current task.

[0029] S3. For the identified target region, based on the two-dimensional visual data of the target region, multi-view... Figure 3 The 3D reconstruction algorithm constructs a local 3D model of the target area, while simultaneously recording the navigation pose data of the underwater robot when constructing each local 3D model.

[0030] In step S2, after confirming the existence of a target region, the algorithm first extracts multiple frames of 2D observation images containing the target region that are temporally continuous and have sufficient overlap in the field of view from the cached continuous image sequence, forming a dedicated multi-view image set for reconstructing the local region. Subsequently, the algorithm automatically processes this image set, including feature point detection and description, cross-image feature point matching, preliminary estimation of camera motion using epipolar geometry, calculation of the 3D spatial coordinates of matching feature points using triangulation principles to generate a sparse point cloud, execution of bundle adjustment for global optimization to simultaneously optimize the 3D point coordinates and camera pose, and finally, dense point cloud matching and surface mesh generation. Through this series of calculations, a high-precision triangular mesh model of the target region and its surrounding small surface area is output, i.e., a local 3D model. When the entire 3D reconstruction calculation process begins, the navigation pose data corresponding to the key frame moments of acquiring the multi-view image set is immediately read and locked from the underwater robot's integrated navigation. This data is typically calculated by fusing inertial measurement unit, Doppler log and global positioning water surface correction unit, and includes the robot’s three-dimensional position and three-dimensional attitude angle in global coordinate system, serving as a reference benchmark for the position and orientation of the local three-dimensional model in macro space.

[0031] For example, upon identifying a suspected corrosion area on the surface of a subsea pipeline, the system immediately retrieves the ten most recent consecutive high-resolution images containing that area from the cache. The 3D reconstruction module first uses a SIFT (Scale-Invariant Feature Transform) feature detector to extract features from these images and performs feature matching between consecutive image pairs. Based on these matching points, the algorithm gradually reconstructs the approximate motion trajectory of the camera and a rough 3D point cloud when these ten images were captured. Then, it performs overall optimization using bundle adjustment to obtain a precise sparse 3D point cloud with scale information. Next, using multi-view stereo vision technology, a dense 3D point cloud is generated based on this sparse point cloud and the optimized camera pose, ultimately constructing a triangular mesh surface model that characterizes the uneven details of the corrosion area. Simultaneously, while processing these images, the robot's core frame pose data during the capture of these ten images is recorded, such as the robot's precise latitude, longitude, depth, roll angle, pitch angle, and heading angle when capturing the first and last frames. The localized erosion mesh model and its associated pose data will be stored as a complete data package and transmitted to the subsequent global modeling process.

[0032] It should be noted that if the surface texture features of the target area are sparse, preventing multi-view reconstruction algorithms from extracting a sufficient number of matching feature points—for example, on a smooth, rusted surface—structured light or binocular stereo vision can be used as auxiliary reconstruction methods. Structured light methods actively project known optically encoded patterns onto the target surface to artificially create high-contrast, precisely identifiable feature points or stripes, thus providing rich, matching visual information for 3D reconstruction. Binocular stereo vision methods, on the other hand, calculate pixel depth directly based on the parallax principle using images simultaneously acquired by two cameras with known relative positions and orientations; its matching process has a lower dependence on natural texture. Both methods can effectively overcome the reconstruction failure problem caused by the lack of features inherent in the target itself.

[0033] S4. After completing the observation of the object under test, construct a three-dimensional reference model of the object under test based on the local three-dimensional models and the navigation pose data corresponding to each local three-dimensional model.

[0034] First, the navigation pose data corresponding to all local 3D models are input into an optimization framework. This pose data itself exhibits drift and inconsistency due to accumulated sensor errors. The optimization framework constructs a pose graph based on the temporal and spatial adjacency relationships of all pose points. In the graph, each pose is treated as a node. If the local 3D models corresponding to two poses have a potential overlapping region in space, a constraint edge is established between these two nodes. The strength of this constraint edge can be estimated by analyzing the geometric consistency of common feature points in the two local 3D models. Subsequently, a nonlinear optimization algorithm, such as graph optimization over Lie algebras, is used to globally adjust the entire pose graph, solving for a set of adjusted pose nodes with optimal global consistency. This process essentially corrects the accumulated errors of the navigation trajectory. Finally, using the optimized globally consistent pose as a spatial skeleton, the 3D coordinates of feature points in all local 3D models are transformed to the same global coordinate system and fused to generate a global 3D point cloud or simplified surface model that covers the entire macroscopic shape of the object under test but has sparse details. This model serves as the 3D reference model required for subsequent fine fusion.

[0035] For example, after completing a circumferential observation of an underwater bridge pier, fifty local 3D models of cracks scattered at different locations on the pier's surface were obtained, each model accompanied by a navigation pose. The global model building module reads these fifty poses and automatically detects which local 3D models may be spatially adjacent. Subsequently, a pose graph containing fifty nodes is constructed, and constraints are added between nodes with estimated overlap. The entire graph is iteratively solved using a Gauss-Newton optimizer, ultimately yielding fifty corrected, accurate poses that eliminate trajectory closure errors. Next, the module uniformly transforms all feature points in these fifty local 3D models to a global coordinate system with the center of the pier's bottom as the origin, based on their corresponding corrected poses. All these feature points are aggregated to form a sparse point cloud depicting the overall outline and main surface features of the pier; this point cloud serves as the 3D reference model of the pier.

[0036] S5. Based on the navigation pose data corresponding to each local 3D model, fill each local 3D model in the 3D reference model to generate a 3D model of the object to be tested.

[0037] The process involves reading the 3D reference model generated in step S4, which defines the global coordinate system and the macroscopic spatial positioning of the object. Next, for each local 3D model to be fused, an initial spatial transformation matrix is ​​calculated using its accompanying navigation pose data. This matrix initially aligns the vertex coordinates of the local 3D model from its own coordinate system to the global coordinate system. Since errors exist in both navigation and reconstruction processes, this initial alignment is imprecise. Therefore, the module performs fine registration on each initially placed local 3D model, using the corresponding surface region in the 3D reference model as a reference. This is typically achieved through an iterative nearest-point algorithm or its variants. This algorithm iteratively calculates the optimal rigid body transformation to minimize the average distance between the point cloud of the local 3D model and the corresponding point cloud on the surface of the reference model, thereby correcting pose deviations. Filling refers to precisely registering and aligning each local 3D model to the unified spatial framework defined by the 3D reference model through geometric transformations. Subsequently, these aligned model data are fused into a seamless, complete, and consistent 3D model.

[0038] After completing the independent and fine registration of all local 3D models, the global fusion process is initiated. The point cloud data of all local 3D models and the data of the 3D reference model are input into a unified spatial data structure, such as a global truncated symbolic distance field. In this field, each spatial voxel accumulates weighted symbolic distance values ​​from all overlapping data. By solving the isosurface of this field, a seamless, smooth, and uniform watertight triangular mesh model can be directly generated. Finally, optional global optimization steps, such as outlier removal based on statistical filtering and anisotropic smoothing of the overall mesh, are used to further improve the model quality, ultimately outputting a complete 3D model of the object under test that integrates macroscopic morphology and local details.

[0039] For example, see Figure 8 As shown, after observing a section of subsea pipeline, the underwater robot obtains a 3D reference model describing the overall cylindrical shape of the pipeline, as well as local 3D models of three corrosion pits attached to different locations on the pipeline surface. The data fusion module first initially attaches the corrosion models to their approximate positions on the pipeline reference model, like patches, based on the robot's pose recorded for each corrosion model. Subsequently, the module initiates a fine registration process for each corrosion patch, allowing the patch's point cloud to undergo minor translations and rotations on the cylindrical surface of the pipeline reference model until it achieves optimal alignment with the reference surface. Once all fifteen patches are precisely in place, the module creates a high-resolution voxel space, simultaneously integrating the point cloud data of the pipeline reference model and all registered corrosion patches, and calculates the average symbolic distance of each voxel to all nearest surfaces. A single isosurface mesh is extracted from this fused voxel field using the moving cube algorithm. This mesh preserves the smooth cylindrical shape of the pipeline while precisely embedding the geometric details of all corrosion pits, forming a complete 3D model of the pipeline that can be directly used for defect quantification analysis.

[0040] See Figure 2 , Figure 2 This is a schematic diagram of the construction process of the local three-dimensional model provided in the embodiments of this application, that is, a schematic diagram of the specific process of step S3.

[0041] S3. Based on the two-dimensional visual data of the target region, multi-view... Figure 3 The 3D reconstruction algorithm constructs a local 3D model of the target region, including: S31. Identify at least two projection markers within the two-dimensional observation image and establish a first reference dimension based on a preset fixed distance between the corresponding transmitters.

[0042] The core of this step is to utilize a pair of transmitters with a known physical distance to provide an absolute scale reference for visual measurement. The transmitters are specialized devices mounted on underwater robots or auxiliary carriers, whose function is to actively form marker points on the surface of the object under test that can be clearly captured by an optical camera. In addition to laser transmitters, various other devices based on physical principles can be used. For example, a hydroacoustic transducer can be used to emit focused sound waves onto the object's surface, creating a sound pressure variation area on the surface through an acoustic lens. This area disturbs water flow or deposits, thus appearing as visually identifiable texture changes. Alternatively, actively emitting beacons or highly reflective passive beacons pre-fixed to the surface of the object under test can be used as marker points, with their spatial positions known. Acousto-optic composite targets can also be used, which emit light upon receiving a specific acoustic signal, forming synchronized acoustic-optical markers. Regardless of the transmitter used, the center distance between the two transmitting units is precisely calibrated during installation or manufacturing and input as a known parameter. In the two-dimensional observation image, image processing algorithms identify the precise pixel coordinates of these two projected marker points. Subsequently, based on the camera's imaging model, known focal length parameters, and the actual physical distance between the two marker points, the conversion ratio between the image's pixel scale and the true physical scale under the current observation posture is calculated using spatial forward intersection or visual measurement principles. The first reference size established thus is a length value with a clear physical meaning. First reference size = (Dpixel / Lpixel) × Lreal or scale factor = Lreal / Lpixel, where Lreal represents the known fixed physical distance between the transmitters corresponding to the two projection marker points, in meters. Lpixel represents the pixel distance calculated from the pixel coordinates of the two projection marker points in the two-dimensional observation image, in pixels. Dpixel represents the pixel distance between any two points in the image to be calculated.

[0043] For example, in turbid underwater environments where laser scattering is severe, a pair of high-frequency underwater acoustic transducers can be used. These two transducers are 50 centimeters apart and emit high-frequency, narrow-beam sound waves. The sound waves generate minute vibrations on the pipe surface, causing attached organisms to temporarily detach, forming two temporary circular areas in the camera's image that differ in color or texture from the surrounding area. Image recognition algorithms identify these two acoustic markers based on their specific size and shape. Given that the actual distance between the two transducers is 50 centimeters, by analyzing the pixel positions of the two markers in the image and the camera's orientation, the actual physical length corresponding to each pixel in the current frame is calculated, thus establishing a first reference dimension.

[0044] In this embodiment, the actively emitted markers (such as laser or acoustic markers) can still be reliably identified in low-contrast environments, providing a stable feature benchmark for reconstruction. The multi-view reconstruction algorithm and depth map fusion strategy can maintain robustness in areas lacking natural textures through auxiliary means. At the same time, real-time fixed-distance observation control effectively reduces image quality fluctuations caused by medium scattering and light attenuation, ensuring the consistency of data acquisition. This guarantees the feasibility and reliability of the entire process from image recognition to 3D model generation in complex underwater environments.

[0045] S32. Select two first feature points within the target area, and use the distance between the two first feature points calculated based on the first reference size as the first size.

[0046] The process involves using an existing physical scale to perform initial scale measurements on natural features within the target area. In images confirmed to contain target areas such as cracks or corrosion, two highly discriminative and stable feature points are manually designated or automatically selected using a feature detection algorithm. These two first feature points can be the endpoints of cracks, corner points of corroded areas, or any locally significant key points. Subsequently, the pixel distance between these two first feature points in the image is measured. Since the first reference size provides a conversion ratio between the "pixel distance" and the "true physical size" in the current image, by applying this ratio to the pixel distance between the first feature points, the actual physical distance between these two feature points on the object's surface can be directly calculated; this calculation result is the first size.

[0047] Specifically, when the target region is classified as a crack region, the binarized image or extracted contour of the region is first thinned to obtain a skeleton line with a single pixel width, accurately representing the central axis direction of the crack. Subsequently, the algorithm traverses this skeleton line, identifying all endpoints with only one adjacent pixel. For simple linear cracks, there are usually two obvious endpoints, which are directly selected as the two first feature points. For complex cracks with branches, the two farthest endpoints of the main branch are selected. The physical significance of this selection strategy is that the two endpoints of the crack define its maximum extension range and are usually the most stable features that are easiest to continuously track from different viewpoints.

[0048] When the target region is classified as a rusted area, its outer closed contour is first extracted. Then, the algorithm employs two parallel or optional strategies to select feature points. The first strategy calculates the Euclidean distance between all polygon vertices (corners) on the contour and selects the pair of vertices furthest apart as the first feature points. This pair of points is called the "diameter endpoints" of the contour and best represents the overall spatial distribution of the rusted region. The second strategy calculates the curvature of each point on the contour curve, identifies several points with the largest local curvature (i.e., corners or tips where the contour direction changes most drastically), and selects two points from these high points that satisfy specific geometric constraints, such as two points that are far apart or have the largest curvature values, as the first feature points. This strategy is suitable for rusted regions with irregular contours but prominent features, where these points with the largest curvature are also stable visual features.

[0049] For example, in the same frame of pipe image where laser spots were identified, a noticeable longitudinal crack exists within the target detection bounding box. The upper and lower endpoints of this crack are automatically selected as two first feature points. The pixel distance between these two endpoints is measured to be 300 pixels. It is known that the previously calculated first reference size of 19.8 cm corresponds to a distance of 250 pixels between the two laser spots in the image. Based on this proportional relationship, the first dimension between the upper and lower endpoints of the crack can be calculated to be approximately 23.76 cm.

[0050] S33. Map the first dimension to three-dimensional space to form a second reference dimension.

[0051] The purpose of this step is to transform a physical dimension measured on a two-dimensional image plane into an absolute dimension in three-dimensional space that directly constrains subsequent 3D reconstruction. Specifically, this can be achieved through triangulation and bundle adjustment, mapping two-dimensional feature point pairs to three-dimensional spatial coordinates, using their Euclidean distance as a second reference dimension. To obtain its true length in three-dimensional space, multi-view geometry is needed for mapping. Figure 3 Angle measurement enables 2D-to-3D mapping. Specifically, in multiple consecutive frames of imagery containing the target region, two first feature points are tracked, and their approximate positions in 3D space are initially estimated using a structure-of-motion reconstruction algorithm. Subsequently, the first dimension calculated from the image is used as a strong constraint on the distance between these two 3D points. Through optimization calculations, their true 3D coordinates in the world coordinate system are precisely derived. The precise Euclidean distance between these two 3D points is then the more general and accurate second reference dimension. The second reference dimension refers to incorporating the first reference dimension calculated based on 2D imagery as a spatial constraint into multi-view... Figure 3 In the optimization process of 3D reconstruction, the final determination of the precise absolute physical distance between two first feature points in the 3D global coordinate system.

[0052] For example, by tracking the movement of the aforementioned crack endpoints across five consecutive frames, preliminary triangulation is used to estimate a rough pair of coordinates in three-dimensional space. At this point, a first dimension of 23.76 cm is added as a fixed constraint to the optimization equation, requiring that the distance between the two optimized three-dimensional points must equal this value. After bundle adjustment optimization, the precise three-dimensional coordinates of the two endpoints are obtained, and the straight-line distance between them is determined to be 23.5 cm. This distance serves as the second reference dimension, a precise and effective physical scale benchmark in global three-dimensional space.

[0053] S34, Combining the second reference dimension and multiple views Figure 3 The 3D reconstruction algorithm constructs a local 3D model.

[0054] This step involves fusing the obtained absolute physical scale reference depth into the standard multi-view. Figure 3 In the 3D reconstruction process, the goal is to generate a local 3D model with the correct scale. When using motion-based structure reconstruction algorithms or similar methods for 3D reconstruction, whether it's sparse point cloud reconstruction or dense surface reconstruction, the entire optimization process is essentially solving for camera parameters and 3D point coordinates. A second reference size, as a known, high-confidence absolute distance observation, is introduced as an additional constraint into the global optimization objective function. During the bundle adjustment optimization stage, this constraint, together with the reprojection error constraint, forces the algorithm to output a 3D model that not only satisfies visual consistency but also meets the known physical dimensions. This directly solves the problem of scale uncertainty in monocular or ordinary multi-view reconstruction, ensuring that the final generated local 3D model has a real-world physical scale from the outset, eliminating the need for subsequent scaling and calibration.

[0055] For example, when performing a complete 3D reconstruction of the crack region, feature points from multiple frames of images are first extracted and matched. In the subsequent global optimization process using bundle adjustment, in addition to minimizing the reprojection error of all feature points, the distance between the 3D point pairs corresponding to the two endpoints of the crack is constrained to be equal to the second reference size of 23.5 cm. Under this dual constraint, the optimization process simultaneously and accurately solves for all camera poses and 3D point coordinates, and automatically determines the correct model scale. The final generated local triangular mesh model of the crack has all dimensions, including crack length, width, and depth, that are real values ​​with direct physical meaning.

[0056] This embodiment establishes the initial physical scale by introducing actively projected marker points and then transfers and constrains it to the 3D reconstruction process based on natural features, thereby directly obtaining a local 3D model with accurate physical dimensions. This effectively solves the problems of scale ambiguity and insufficient measurement accuracy in purely visual 3D reconstruction in underwater environments.

[0057] See Figure 3 As shown, Figure 3 This is a flowchart illustrating the process of constructing a local three-dimensional model based on a second reference dimension, specifically a flowchart illustrating step S34, as provided in an embodiment of this application.

[0058] S34, Combining the second reference dimension and multiple views Figure 3 3D reconstruction algorithms construct local 3D models, including: S341a. Based on the second reference size, scale calibration and distortion correction are performed on the multi-view images acquired during the operation of the underwater robot to confirm a unified scale benchmark.

[0059] First, the second reference dimension determined in step S33 is read. This dimension represents a verified absolute physical length in three-dimensional space. This length corresponds to the actual distance between a pair of specific feature points that can be traced in the multi-view image sequence. Based on this known real-world distance and the pixel distance between these feature points in any frame of the reference image, the precise scale factor of the image at the current imaging position, i.e., the physical dimension represented by each pixel, can be directly calculated. Simultaneously, the camera's internal parameters, particularly the radial and tangential distortion coefficients of the lens, obtained through pre-calibration in the laboratory, are used to perform geometric correction on each input multi-view original image, eliminating image distortion caused by lens optical defects. The image set after distortion correction and given a precise physical scale factor constitutes a scale-consistent and geometrically accurate input dataset, providing a unified scale benchmark and a high-quality image foundation for subsequent three-dimensional calculations.

[0060] For example, the second reference size was determined to be a real distance of 15.2 cm between two points. In the selected keyframe image, the pixel distance between these two corresponding points was measured to be 380 pixels. From this, the scale factor of the image can be directly calculated to be 0.04 cm per pixel under the current imaging conditions. Subsequently, using known camera distortion parameters, all 50 original images used for reconstruction were corrected to correct the barrel distortion caused by the fisheye lens. After this processing, all images not only eliminated optical distortion, but each image also had a clear physical scale meaning, that is, any pixel distance in the image can be proportionally converted to a real physical distance based on its image location.

[0061] S342a. Extract the second feature points from the images of each viewpoint, match the second feature points between different viewpoints, and establish the correspondence between viewpoints.

[0062] In this process, for calibrated multi-view images, a feature point detection algorithm scans each image to identify local structures with significant uniqueness that may be repeatedly observed from different viewpoints; these structures are defined as second feature points. Commonly used detection algorithms include scale-invariant feature transformation or accelerated segmented test features. For each detected second feature point, the algorithm calculates a high-dimensional descriptor vector to digitally represent the image appearance information surrounding that point. Subsequently, by comparing the similarity between feature point descriptor vectors in different images, feature matching is performed across all viewpoint images to find multiple image projection points belonging to the same physical point in space. This matching process typically employs a nearest neighbor search strategy, supplemented by geometric constraints such as cross-validation to eliminate incorrect matching pairs, ultimately forming a network composed of a large number of correctly matched point pairs, thereby firmly establishing the correspondence between images from different viewpoints.

[0063] For example, SIFT feature point detection is performed on each of 50 pipe surface images that have undergone scale and distortion correction. In one image, the algorithm may extract approximately 2000 feature points, such as the edge of a crack, near a rust spot, and in the texture of a smooth background, and calculate a 128-dimensional descriptor for each point. Subsequently, a fast KD-Tree (K-Dimensional Tree) search is used to compare the similarity of the descriptor of each feature point in this image with the descriptors of all feature points in the other 49 images, finding the best match for each point. For example, a feature point at the crack endpoint in image A is found to have highly similar descriptors in images B, C, and D, and the relative geometric positions of these matching points conform to epipolar constraints. This confirms that these four pixels are projections of the same crack endpoint in space from four different viewpoints, thus establishing a stable correspondence across four images.

[0064] S343a. Calculate the three-dimensional coordinates of the matched second feature points using triangulation to generate an initial sparse point cloud.

[0065] For each pair of cross-view matching points established in step S342a, their pixel coordinates in different images are known, along with the camera's internal parameters and estimated external pose parameters when these images were captured. According to epipolar geometry, observation rays from two different viewpoints should intersect at their corresponding real physical points in three-dimensional space. Triangulation can be used to determine the coordinates of the most likely intersection point in space. When a matching point appears in more than two views, more observation data can be used to find an optimal three-dimensional point coordinate using the least squares method, minimizing the sum of reprojection errors from that point to all corresponding imaging rays. Performing this calculation on thousands of successful matching point pairs yields a series of discrete sets of three-dimensional point coordinates distributed in three-dimensional space. Although this set only contains the locations of some salient features in the scene, it outlines the basic spatial structure of the target area and is called the initial sparse point cloud.

[0066] For example, for the crack endpoint feature points that were successfully matched in images A, B, C, and D, a preliminary motion estimation algorithm was used to estimate the approximate position and orientation of the camera relative to the global coordinate system when these four images were captured. Using the projection matrices of these four cameras and the precise pixel coordinates of the feature points in the four images (e.g., A(320,240), B(305,245), C(310,235), D(315,238)), an overdetermined system of equations was constructed. An optimal 3D point coordinate system (e.g., X=1.05m, Y=0.30m, Z=2.01m) was solved using linear triangulation or nonlinear least squares optimization methods. Performing the same calculation on all thousands of similar matching points within the reconstructed area generated a sparse point cloud composed of thousands of such 3D points, providing a preliminary 3D outline of the crack and corrosion region.

[0067] S344a. Based on the actual spacing of the second reference size, perform scale scaling calibration on the initial sparse point cloud.

[0068] In the standard motion-to-structure reconstruction process, the scale of the reconstructed sparse point cloud and camera trajectory is arbitrary and disproportionate to the real world. Although a scale factor is assigned to a single image in step S341a, a strong constraint is still needed in global optimization to unify the scale of the entire reconstruction model. Here, the actual physical distance of the second reference size is used as this global scale constraint. Specifically, in the initial sparse point cloud, the two 3D points corresponding to the second reference size are located. The Euclidean distance between these two 3D points at the current reconstruction scale is calculated; this distance is a dimensionless value. Subsequently, the scaling factor between this calculated distance and the actual physical distance of the second reference size is calculated. Finally, this scaling factor is applied to the entire sparse point cloud, performing a uniform scaling transformation on the coordinates of all 3D points in the point cloud. After this operation, the entire sparse point cloud is accurately scaled to a physical scale consistent with the real world.

[0069] For example, the actual physical spacing of the second reference dimension is 15.2 cm. In the uncalibrated initial sparse point cloud, the calculated distance between the coordinates of two corresponding 3D points is 1.52 arbitrary units. Therefore, the scale factor of the current reconstructed model is: 1 arbitrary unit corresponds to 10 cm (15.2 cm / 1.52). Then, the X, Y, and Z coordinates of all points in this sparse point cloud are multiplied by a scale factor of 10, changing the unit to centimeters. After scaling, the distance between the two corresponding points is precisely adjusted to 15.2 cm, and the entire point cloud model is synchronously scaled to a physical size completely consistent with the real object.

[0070] S345a. The calibrated sparse point cloud is densified, and a local 3D model is generated through mesh reconstruction.

[0071] First, multi-view stereo vision dense matching is performed using calibrated sparse point clouds and precise camera poses as input. This process estimates the depth or 3D position of each calibrated image pixel-by-pixel or patch-by-pattern on the surface of the object it covers, generating a depth map corresponding to each image. These depth maps from multiple views are then fused into a unified 3D space. Due to redundancy and noise between depth maps, the fusion process uses a voting or weighted averaging mechanism to generate a single, more accurate, and complete dense 3D point cloud. Finally, based on this dense point cloud, surface reconstruction algorithms, such as mesh generation methods based on Delaunay triangulation or Poisson reconstruction, are used to infer the topological connectivity of the surface, constructing a watertight, renderable local 3D model expressed in the form of a triangular mesh. This model not only possesses fine surface geometric details, but all its dimensions retain the true physical scale inherited from the calibration in step S344a.

[0072] For example, using a scale-calibrated sparse point cloud and 50 camera poses, a patch-based multi-view stereo matching algorithm is initiated. The algorithm calculates a depth map for each image, where each pixel value represents the distance from the camera to that point on the object's surface. For instance, for a crack region, the depth map clearly shows that the crack grooves are deeper than the surrounding surface. These 50 depth maps are then back-projected into 3D space and fused into a dense point cloud containing millions of 3D points, finely depicting the unevenness of the corrosion and the direction of the cracks. Finally, the dense point cloud is processed using a Poisson surface reconstruction algorithm to generate a smooth, closed triangular mesh surface. This mesh model can be directly used to measure the true width and depth of the cracks and the true volume of the corrosion region.

[0073] See Figure 4 As shown, Figure 4 This is another flowchart illustrating the construction of a local three-dimensional model based on a second reference dimension, specifically another flowchart illustrating step S34, which is an embodiment of this application.

[0074] S34, Combining the second reference dimension and multiple views Figure 3 3D reconstruction algorithms construct local 3D models, including: S341b, Use two first feature points within the target area as spatial fixed reference anchor points.

[0075] After the target area is identified, two highly significant and repeatable physical feature points are selected from the two-dimensional visual data of that area. These two first feature points are typically such as the endpoints of cracks, corners of rusted areas, or the centers of specific texture patches, which can be clearly identified and stably tracked in multiple consecutive images. These two points are designated as fixed spatial reference anchor points in this round of local 3D reconstruction. Their core value lies in the fact that, on the one hand, they possess visual features that can be extracted by image algorithms, and on the other hand, the actual physical distance between them has been calculated and confirmed as a second reference dimension through the aforementioned steps, thus serving as a reliable bridge connecting image pixel information and real-world physical dimensions.

[0076] For example, when modeling a corrosion area on the surface of a subsea pipeline, a sharp protrusion in the upper left corner and a noticeable pit in the lower right corner are selected as two primary feature points. These two points are actually ten centimeters apart on the pipeline surface. These two points are registered as fixed reference anchor points for this reconstruction and will subsequently appear in all images of the corrosion area from different perspectives, serving as a common benchmark for all geometric calculations.

[0077] S342b: Based on the pixel correspondence of two spatially fixed reference anchor points in multi-view images, and combined with multi-view geometric constraints, the epipolar geometric relationship of multiple view images is corrected to ensure the accuracy of spatial correspondence between different view images.

[0078] First, in all multi-view images containing the target region, the pixel positions of two spatially fixed reference anchor points are accurately detected and matched. Since the true correspondence between these two points is known and certain, they constitute a set of strongly constrained matching point pairs. Based on these matching point pairs, and combined with epipolar geometry principles, the fundamental matrix or essential matrix between each pair of images can be accurately calculated and optimized. This process essentially corrects geometric errors caused by inaccurate camera pose estimation or image feature matching noise. By incorporating the matching relationship of these two anchor points as a reliable constraint into the global optimization, such as by assigning a high weight in bundle adjustment, the overall consistency of relative pose estimation across all camera views can be significantly improved, thereby ensuring that scene points observed from different views are strictly corresponding geometrically.

[0079] For example, eight images of the corroded area are collected from eight different angles. In each image, a feature matching algorithm is used to find the pixel coordinates of the two reference anchor points (protrusions and pits). Using the image pairs formed by the eight images and the matching coordinates of the two anchor points in each pair, an optimal set of fundamental matrices is calculated. This optimization process forces the epipolar geometry of all images to be adjusted so that the projections of the two anchor points in all image pairs satisfy the epipolar constraints as much as possible, thereby significantly improving the intrinsic accuracy of the multi-view geometric model.

[0080] S343b: Perform depth estimation on the corrected images from each viewpoint to generate a multi-view depth map.

[0081] In this process, using the optimized camera parameters and multi-view images from step S342b, the depth value of each pixel in each individual view image is estimated. Depth estimation can employ a stereo matching principle, where the image is used as a reference view, and corresponding pixels are searched in other adjacent views to calculate depth through disparity; alternatively, a deep learning-based method can be used, where a neural network directly regresses depth from single or paired images. Regardless of the specific algorithm used, because the geometric relationships of the input multi-view images have been optimized and corrected, the correspondence search between different views is more accurate, thus generating higher-quality and more consistent single-view depth maps. Finally, a set of depth maps corresponding one-to-one with the original multi-view images is obtained, where each pixel value in each depth map represents the distance from the camera at that viewpoint to the corresponding point on the object's surface.

[0082] For example, for the eight corrected images mentioned above, a multi-view stereo algorithm based on patch matching is employed. Using the first image as a reference, the algorithm makes assumptions about the depth of each pixel within the pixel grid of the eroded area it covers. It then calculates the similarity between this assumed pixel block and the corresponding epipolar pixel blocks in the second to eighth images to find the best match, thereby determining the depth of that pixel. This process generates a depth map for the first image. Similarly, depth maps are generated for the remaining seven images, each describing the three-dimensional shape of the eroded surface from a different perspective.

[0083] S344b: Based on the second reference size, compare the calculated distance and actual spacing between the two spatially fixed reference anchor points in each depth map, and calibrate the scale accuracy of each depth map through a reverse iterative optimization algorithm.

[0084] While depth maps contain 3D information, the scale represented by their values ​​may be uncertain or biased. For each generated depth map, the pixel positions of two spatially fixed reference anchor points in the image are located, and the depth values ​​at these two pixels are read from the depth map. Combined with camera intrinsics, these two pixel positions and their depth values ​​can be back-projected into 3D space, calculating the spatial distance between the two anchor points in the local 3D coordinate system defined by this depth map—this is a calculated distance. Subsequently, this calculated distance is compared with the known real physical distance, which serves as a second reference size, to calculate a scale correction factor. To achieve optimal global consistency, a backward iterative optimization algorithm can be used. This algorithm aims to minimize the overall deviation between the calculated distances of anchor points and the real physical distances across all depth maps, iteratively adjusting a scale scaling parameter (and sometimes a slight offset parameter) for each depth map until all depth maps are calibrated to a uniform physical scale consistent with the real world.

[0085] For example, the true distance between two reference anchor points is known to be 10 centimeters. In the depth map of the first image, the backprojected distance between the two points is 9.5 centimeters; in the depth map of the second image, the calculated distance is 10.2 centimeters. This indicates a slight difference in scale between the two depth maps. An optimizer is started, assigning a scale parameter s to each depth map. The objective function is to minimize the sum of squares of the differences between the adjusted distance (calculated distance multiplied by s) and the true distance of 10 centimeters for all eight depth maps. After several iterations, the optimizer solves for a set of optimal scale parameters, for example, s1 is approximately 1.053 and s2 is approximately 0.98. After applying these parameters to adjust all depth values ​​in the eight depth maps, the backprojected distance between the two anchor points in each map is extremely close to 10 centimeters, thus achieving precise physical scale calibration of the depth maps.

[0086] S345b: The calibrated multi-view depth map is fused to generate a dense point cloud that conforms to the scale of the actual scene.

[0087] Each calibrated depth map is back-projected onto the same global 3D coordinate system based on its corresponding precise camera pose, generating a 3D point cloud. Due to overlapping observations from multiple perspectives, the same object surface may be observed from different angles by multiple depth maps, resulting in numerous repetitive but potentially noisy points in 3D space. The fusion process aims to merge this redundant information and resolve potential observation conflicts. A common fusion method is to construct a global truncated signed distance field (TSDF). The 3D space is discretized into a voxel grid, with each voxel storing a signed distance value to the nearest surface and a weight. All points generated from the depth maps are iterated through, and each point is updated with a weighted average of distances and weights to its neighboring voxels. The more accurately the depth maps are calibrated, the higher the consistency of data from different sources within the TSDF field. Finally, by extracting the zero isosurface of this TSDF field, a seamless, dense 3D triangular mesh model can be directly obtained; alternatively, all back-projected point clouds can be merged and filtered to obtain a dense point cloud that conforms to the actual scale.

[0088] For example, eight scale-calibrated depth maps are back-projected based on their respective camera poses to generate eight 3D point clouds. These eight point clouds are then used to update a global TSDF voxel field with a resolution set to 0.5 millimeters. For the same erosion pit in space, the point cloud data from the eight perspectives collaboratively determine the precise location and shape of the pit's surface. Finally, a moving cubes algorithm is used to extract a triangular mesh from this TSDF field that integrates all the information. This mesh model is not only rich in detail, but its dimensions, such as the diameter and depth of the pit, are also physically meaningful values.

[0089] S346b: After denoising the dense point cloud, a local 3D model is constructed.

[0090] Dense point clouds or meshes obtained directly from depth map fusion may contain outliers, surface irregularities, or small holes. First, a statistical filter is applied to remove isolated noise points far from the main point cloud group. Then, a conditional filter or radius filter may be used to further smooth the data while preserving feature edges. Based on point cloud optimization, if a point cloud was previously generated, a mesh is constructed using a surface reconstruction algorithm; if a mesh has already been generated, mesh smoothing and hole repair are performed. Surface reconstruction algorithms, such as Poisson reconstruction, can robustly generate closed, watertight triangular mesh models from point clouds and their normal information. The final output local 3D model is a detailed, smooth, scale-accurate, and geometrically complete digital 3D entity.

[0091] For example, the initial mesh extracted from the TSDF field may contain some floating noise in the background region. Statistical filtering is used to remove points with fewer than a threshold of neighboring points. Next, a Laplacian smoothing algorithm is applied to the mesh to make the surface smoother without excessively blurring the corrosion edges. For tiny holes caused by occlusion during reconstruction, a mesh-based hole-filling algorithm is used for repair. Finally, a local 3D mesh model with metrological-grade accuracy is obtained, accurately depicting the geometry of the corrosion region and directly usable for calculating corrosion volume or maximum depth.

[0092] This approach embeds the depth of spatial anchor points with known absolute dimensions into the entire chain from multi-view geometry optimization and depth map generation to scale calibration and fusion. This enables a high-precision 3D reconstruction process with depth maps as the core and a posteriori physical scale calibration as a guarantee. It effectively solves the scale uncertainty problem that is common in learning-based depth estimation methods in complex underwater environments, and ultimately produces a local 3D model with realistic scale and rich details.

[0093] In one possible embodiment, real-time identification of the two-dimensional observation image to confirm the existence of the target area includes: At least one target region is identified from a two-dimensional observation image using a pre-trained target detection model; wherein the target region is a region whose disease index meets the preset feature threshold conditions, the feature threshold conditions including: crack length is greater than a preset length threshold; and / or corrosion area is greater than a preset area threshold.

[0094] The study employed a deep learning model trained and optimized using a heavily annotated dataset of underwater structural defects. This model was integrated into the embedded edge computing unit of the underwater robot. Upon input of two-dimensional observation images, the model first performs pixel-level semantic segmentation or generates a series of candidate bounding boxes, from which all suspected crack or corrosion areas are preliminarily identified.

[0095] For each initially identified candidate region, the model or its associated post-processing algorithm further calculates its key deterioration indicators. Deterioration indicators are parameters used to quantitatively assess the severity of specific types of defects on the surface of underwater structures, such as crack length, corrosion area, or a combination thereof. For crack regions exhibiting linear characteristics, this indicator is its projected length on the 2D image plane; for corrosion regions exhibiting planar characteristics, this indicator is its projected area on the 2D image plane. Pre-defined, engineering-validated feature thresholds are used, such as setting the crack length threshold to 10 centimeters and the corrosion area threshold to 1 square decimeter. The algorithm compares the calculated deterioration indicators for each candidate region with the corresponding preset thresholds in real time. Only when the quantified indicators of a region, such as its calculated length being greater than 10 centimeters or its calculated area being greater than 1 square decimeter, is that region ultimately confirmed as a "target region" requiring subsequent detailed 3D modeling.

[0096] The determination of preset length thresholds and preset area thresholds is based on the allowable defect limits specified in the design specifications, safe operation procedures, or industry maintenance standards of the engineering structure to which the object under test belongs, and is comprehensively set in combination with historical inspection data and engineering experience. Specifically, by referring to the technical requirements of relevant industry standards (such as the inspection procedures for underwater structures such as bridges, ports, and water conservancy and hydropower) for indicators such as the allowable width and length of cracks and the allowable range of corrosion, these are converted into pixel length or area thresholds at the image scale, or set according to the empirical values ​​of typical defects that are identified as needing to be focused on and recorded in actual engineering projects, so as to ensure that the identified target areas are all defect areas that have practical evaluation significance in terms of structural safety or maintenance.

[0097] The target detection model consists of a feature extraction backbone network based on a convolutional neural network, a region proposal network, and a classification and bounding box regression head. Its training process uses an underwater structure image dataset containing labeled diseased areas. Features are extracted and region proposals are generated through forward propagation. Then, a multi-task loss function is used to simultaneously optimize the classification accuracy and bounding box localization accuracy. The network weights are iteratively updated through backpropagation and gradient descent algorithms until the model's recognition performance on the test set reaches a predetermined index. This results in a dedicated model that can automatically and accurately identify diseased areas that meet the feature threshold conditions from complex underwater images.

[0098] For example, an underwater robot runs a lightweight segmentation model based on an improved U-Net (U-Shaped Network Architecture) architecture, which has been trained on thousands of images annotated with underwater pipe cracks and corrosion. When the robot captures an image of a concrete pile wall, the model quickly outputs a segmentation result image, highlighting two suspected crack lines and a whole suspected corrosion patch. Subsequently, for the two suspected cracks, the central axis is extracted using a skeletonization algorithm and its pixel length is calculated; for the corrosion patch, the total number of highlighted pixels is directly calculated to obtain the pixel area. Based on pre-stored camera calibration parameters and the current observation distance, these pixel values ​​are converted into physical dimensions. The calculation shows that the first crack is 8 centimeters long, the second is 15 centimeters long, and the corrosion patch area is 0.8 square meters. Comparing these three values ​​with preset thresholds (length 10 centimeters, area 1 square decimeter), it is determined that the second crack (15 centimeters > 10 centimeters) and the corrosion patch (0.8 square meters > 0.01 square meters) meet the feature threshold conditions. Therefore, the two target areas were finally identified and their location information was recorded, triggering the subsequent local 3D modeling process for these two specific areas. The eight-centimeter crack was filtered out and did not occupy subsequent processing resources.

[0099] See Figure 5 As shown, Figure 5 This is a flowchart illustrating the process of generating a three-dimensional reference model provided in an embodiment of this application, specifically a flowchart of step S5.

[0100] S5. After completing the observation of the object under test, based on each local 3D model and the corresponding navigation pose data, construct a 3D reference model of the object under test, including: S51a. Based on the navigation pose data corresponding to each local 3D model, determine the spatial distribution information of each local 3D model in the global coordinate system. The spatial distribution information includes at least its projection contour on the horizontal reference plane and its depth information.

[0101] The underwater robot first reads the navigation pose data associated with each local 3D model, which provides the model's precise position and attitude in the global geodetic coordinate system. Based on this, the coordinates of all vertices of each local 3D model are uniformly transformed to the same global coordinate system based on the horizontal plane through rigid body transformation. In this unified horizontal global coordinate system, geometric analysis is performed on each local 3D model that has undergone coordinate transformation. First, all vertices are vertically projected onto the horizontal reference plane, generating a polygonal region composed of the projected points. This polygon is the horizontal projection contour of the local 3D model, representing the model's two-dimensional occupancy range on the horizontal plane. Second, the vertical coordinate values ​​of all vertices of the local 3D model are analyzed, and their maximum and minimum values ​​are extracted. These two values ​​together define the depth range of the model in the vertical direction, i.e., depth information. Finally, each local 3D model is transformed into a set of concise geometric descriptors with global spatial relationships, namely its projection contour on the horizontal plane and its corresponding depth interval.

[0102] Suppose an underwater robot observes a bridge pier, generating twenty local triangular mesh models, each representing a different crack. Each model includes a navigation pose recording its observation position. First, using this pose data, all vertices of the triangular meshes in all twenty models are transformed into a global coordinate system with the docking station as the origin and the east-north-sky direction as the coordinate axes. Then, for model A describing a crack at the bottom of the bridge pier, all its vertices are ignored (Z-coordinates are ignored) and projected onto a horizontal plane, forming an approximately rectangular point cloud. The convex hull algorithm is then used to generate the circumscribed polygon of this point cloud as its projected contour. Simultaneously, the Z-coordinates of all vertices of model A are calculated, with a minimum value of -10 meters and a maximum value of -9.8 meters, thus providing depth information from -10 meters to -9.8 meters. This process is repeated for all twenty models, resulting in twenty globally aligned projected contours and their depth ranges.

[0103] S52a. Based on the spatial distribution of the projected contours of all local 3D models in the global coordinate system, fit a continuous 2D geometric shape as the inferred horizontal cross section of the object to be measured.

[0104] In this process, the projected contours of all local 3D models are placed within the same horizontal reference plane for global analysis. Since these contours originate from different parts of the object's surface, they may be separate, partially overlapping, or nested. First, the outer boundary vertices of all these contours are merged into a single 2D point set. Then, a shape fitting and synthesis algorithm is used to process this point set. A common approach is to use the Alpha-Shape algorithm, which, by defining a rolling circle radius, can extract potentially non-convex closed polygons that characterize the overall shape of the object from a discrete point set. Another approach is to perform principal component analysis on the point set to determine its main extension directions and fit a standard geometric shape such as a rectangle or ellipse. The continuous 2D geometry generated by this type of algorithm can smoothly encompass most of the key feature points of the local projected contours, thus reasonably inferring the overall cross-sectional shape of the object on the horizontal plane; this shape is the inferred horizontal cross-section.

[0105] Continuing the previous example, all twenty projected contour polygons corresponding to the twenty crack models are superimposed and displayed on a horizontal plane. These contours are scattered across the pier surface, but overall present a roughly circular distribution. All vertices of these twenty polygons are extracted to form a dense two-dimensional point cloud. Using the Alpha-Shape algorithm to process this point cloud, with appropriate radius parameters set, the algorithm automatically generates a smooth, closed polygon that can enclose all points and approximates the actual circular cross-section of the pier. This polygon is the inferred horizontal cross-section of the pier, reflecting the overall horizontal position of the pier more completely than any local contour.

[0106] S53a. Based on the inferred depth information of the horizontal cross section and each local three-dimensional model, a three-dimensional reference model is constructed along the vertical direction.

[0107] First, a global depth range is determined from the depth information of all local 3D models, namely the minimum and maximum depth values ​​among all depth intervals. Then, the inferred horizontal cross-section obtained in step S52a is used as the cross-sectional shape. The construction process involves using this cross-section as the base and vertically stretching or sweeping along a direction perpendicular to the horizontal reference plane, from the global minimum depth value to the global maximum depth value. This process generates a unified columnar or generalized prism in 3D geometry. This 3D volume can be directly expressed as a solid model or represented by its surface mesh. This generated model is the 3D reference model, defining a continuous and complete spatial volume, clearly presenting the macroscopic spatial envelope of the object under test, and providing a rigid spatial reference framework for the subsequent precise placement and fusion of local 3D models.

[0108] For example, by summarizing the depth information of twenty crack models, the global minimum depth was found to be -15 meters, and the global maximum depth to be -5 meters. Next, the circular inferred horizontal cross-section of the pier obtained in the previous step was used as the base shape. This circular cross-section was then stretched vertically from -15 meters to -5 meters, generating a 3D mesh of a vertical cylinder with a height of 10 meters. This cylinder represents the best estimate of the overall spatial occupancy of the pier and is used as the 3D reference model. All detailed local models of the cracks will be positioned and fused using the surface of this cylinder as a reference in subsequent steps.

[0109] See Figure 6 As shown, Figure 6 This is another schematic diagram of the process for generating a three-dimensional reference model provided in the embodiments of this application, namely another specific flowchart of step S5.

[0110] S5. After completing the observation of the object under test, based on each local 3D model and the corresponding navigation pose data, construct a 3D reference model of the object under test, including: S51b. Based on the navigation pose data corresponding to each local 3D model, determine the spatial distribution information of each local 3D model in the global coordinate system. The spatial distribution information includes at least its projection profile on the reference plane and its corresponding normal distance information.

[0111] S52b. Based on the spatial distribution of all projected profiles on the reference plane, fit a continuous two-dimensional geometric shape as the inferred two-dimensional cross-section of the object to be measured.

[0112] S53b: Based on the inferred two-dimensional cross-section and the normal distance information corresponding to each local three-dimensional model, a three-dimensional reference model is constructed along the normal of the reference datum plane.

[0113] In step S51b, the primary task is to determine a reasonable reference plane. This plane does not necessarily have to be horizontal; rather, it can be dynamically defined based on the overall geometric principal direction or main observation direction of the object under test. That is, the reference plane can be horizontal, vertical, or any other plane at any angle. For example, it can be selected as the reference plane by performing plane fitting on the robot's observation path or by performing principal component analysis on the vertices of all local 3D models, and then selecting the plane spanned by the first and second principal components. This plane is uniquely defined in the global coordinate system by an origin and a normal vector. After determining the reference plane, all vertices of each local 3D model are transformed to the global coordinate system using the navigation pose data corresponding to each local 3D model. Subsequently, the vertices of each local 3D model are projected perpendicularly onto the reference plane along the normal direction of the reference plane. The outer boundary of the resulting set of projected points constitutes the projected contour of the model. For example: See [link to relevant documentation]. Figure 7As shown, the four local 3D models are projected onto the reference plane to form four projected profiles, namely the projected profiles of local 3D models 1 to 4. Simultaneously, the signed perpendicular distances from all vertices of each local 3D model to the reference plane are calculated. The statistical characteristics of these distance values, such as the average, minimum, and maximum values, together constitute the normal distance information describing the spatial position of the model relative to the reference plane.

[0114] For example, an underwater robot observes along an inclined seabed pipeline whose axis is not horizontal. Principal component analysis is performed on all vertices of the local 3D model to obtain an inclined plane that best represents the pipeline's orientation, serving as a reference plane. For a local 3D model of observed flange corrosion, the model vertices are first transformed to global coordinates using its navigation pose. Then, all vertices are vertically projected onto the inclined reference plane, forming an approximately circular projection profile. Next, the vertical distance from each vertex of the corrosion model to the reference plane is calculated. The average distance is 0.3 meters, the minimum is 0.25 meters, and the maximum is 0.35 meters. This data represents the normal distance information of the local 3D model relative to the reference plane.

[0115] In step S52b, the goal is to infer the overall two-dimensional projection shape of the target object on the reference plane based on the footprints of all local observation points. The projected contours of all local 3D models obtained in step S51b are placed in a two-dimensional coordinate system defined by the same reference plane for global examination. These contours may be scattered and overlapping, collectively outlining the projection range of the object on this plane. The boundary points of all these contours are extracted to form a dense two-dimensional point set. To synthesize a continuous, complete, and potentially smooth two-dimensional shape from the discrete point set, a shape reconstruction or fitting algorithm is required. Common methods include calculating the convex hull of the point set to obtain the outermost convex polygon, or using the Alpha-Shape algorithm to generate more complex non-convex polygons that capture concave features by controlling the radius parameter. For targets with regular shapes, the least squares method can also be used to fit the point set into standard geometric shapes such as ellipses and rectangles. The continuous closed two-dimensional shape generated through this process is a reasonable inference of the overall cross-section of the target object on the reference plane.

[0116] Continuing the previous example, dozens of local 3D models (representing corrosion, attached organisms, etc.) obtained along the inclined pipe are projected onto an inclined reference plane. These contour points are scattered within a long, narrow region. After merging all contour points, the Alpha-Shape algorithm is used for processing. By setting an appropriate radius parameter, the algorithm automatically connects the outer points, generating a smooth, closed 2D polygon that encloses all internal points and has a shape approximately like an elongated ellipse. This polygon is then inferred as the 2D cross-sectional shape of the inclined pipe on the reference plane.

[0117] In step S53b, the two-dimensional cross-sectional shape and depth range information are combined to construct a three-dimensional volume along the correct direction. First, the normal distance information of all local three-dimensional models is integrated to determine a global normal distance range, which is the minimum and maximum distance of all models to the reference plane. This range defines the spatial span of the object in the direction perpendicular to the reference plane. Then, the inferred two-dimensional cross-section obtained in step S52b is used as the cross-sectional shape. The construction process involves translating and sweeping this two-dimensional cross-section along the normal direction of the reference plane from the global minimum normal distance value to the global maximum normal distance value. This operation generates a unified columnar or generalized prism in three-dimensional space, whose sides are formed by stretching the two-dimensional cross-sectional contour along the normal direction. This three-dimensional volume is the final three-dimensional reference model, establishing a continuous reference coordinate system aligned with the macroscopic spatial posture of the target object, providing a stable basic framework for the subsequent accurate registration and fusion of various local fine models in a unified space.

[0118] For example, analyzing the normal distance information of all local 3D models reveals a global minimum distance of zero meters and a maximum distance of 1.5 meters. Using the approximate elliptical 2D cross-section obtained in the previous step as the basic shape, this elliptical cross-section is swept along the normal direction of the reference plane from zero meters to 1.5 meters, generating a 3D mesh model of a tilted elliptical cylinder with a height of 1.5 meters. The central axis of this elliptical cylinder is perpendicular to the reference plane, and its shape and orientation are consistent with the macroscopic orientation of the tilted pipe. This elliptical cylinder serves as the 3D reference model. All local 3D models addressing various defects on the pipe surface will subsequently be precisely aligned and integrated using this model's surface as a spatial reference.

[0119] Please see Figure 9 As shown, in one embodiment, a 3D modeling device for underwater targets is provided, the device comprising: The acquisition module 901 is used to control the underwater robot to perform fixed-distance observation of the object to be measured according to a preset navigation path, and to acquire two-dimensional observation images in real time through a camera device during the observation process. The identification module 902 is used to identify the two-dimensional observation image in real time to confirm the existence of the target area; Construction module 903 is used to, based on the two-dimensional visual data of the identified target region, employ multi-view... Figure 3 The 3D reconstruction algorithm constructs a local 3D model of the target area, and simultaneously records the navigation pose data of the underwater robot when constructing each local 3D model; The construction module 903 is further configured to, after completing the observation of the object under test, construct a three-dimensional reference model of the object under test based on each of the local three-dimensional models and the navigation pose data corresponding to each of the local three-dimensional models; The generation module 904 is used to fill the three-dimensional reference model with the navigation pose data corresponding to each of the local three-dimensional models to generate a three-dimensional model of the object to be tested.

[0120] In one possible embodiment, the two-dimensional visual data based on the target region is used with multi-view... Figure 3 The 3D reconstruction algorithm constructs a local 3D model of the target region, including: Identify at least two projection markers within the two-dimensional observation image and establish a first reference dimension based on a preset fixed spacing between the corresponding transmitters; Two first feature points are selected within the target area, and the distance between the two first feature points is calculated based on the first reference size and used as the first size. The first dimension is mapped to three-dimensional space to form a second reference dimension; Combining the second reference size and the multi-view Figure 3 The local 3D model is constructed using a 3D reconstruction algorithm.

[0121] In one possible embodiment, selecting two first feature points within the target area includes: If the target area is a crack area, select two endpoints on the geometric contour of the crack as the two first feature points; If the target area is a rusted area, then the two farthest corner points or the points with the greatest curvature on the closed contour of the rusted area are selected as the two first feature points.

[0122] In one possible embodiment, the combination of the second reference size and the multi-view Figure 3 The 3D reconstruction algorithm constructs the local 3D model, including: Based on the second reference size, scale calibration and distortion correction are performed on the multi-view images acquired during the operation of the underwater robot to confirm a unified scale benchmark. The second feature points in each of the aforementioned viewpoint images are extracted, and the second feature points between different viewpoints are matched to establish a correspondence between viewpoints; The three-dimensional coordinates of the matched second feature point are calculated using triangulation to generate an initial sparse point cloud. Based on the actual spacing of the second reference size, the initial sparse point cloud is scaled and calibrated. The calibrated sparse point cloud is densified, and the local 3D model is generated by mesh reconstruction.

[0123] In one possible embodiment, the combination of the second reference size and the multi-view Figure 3 The 3D reconstruction algorithm constructs the local 3D model, including: Use two of the first feature points within the target area as spatial fixed reference anchor points; Based on the pixel correspondence of the two spatial fixed reference anchor points in the multi-view images, and combined with the multi-view geometric constraints, the epipolar geometric relationship of the multiple view images is corrected to ensure the accuracy of spatial correspondence between different view images. Depth estimation is performed on each of the corrected viewpoint images to generate a multi-view depth map; Based on the second reference size, the calculated distance and actual spacing between the two spatial fixed reference anchor points in each depth map are compared, and the scale accuracy of each depth map is calibrated by a reverse iterative optimization algorithm. The calibrated multi-view depth maps are fused to generate a dense point cloud that conforms to the scale of the actual scene. After denoising the dense point cloud, the local 3D model is constructed.

[0124] In one possible embodiment, the real-time identification of the two-dimensional observation image to confirm the existence of the target area includes: At least one target region is identified from the two-dimensional observation image using a pre-trained target detection model; wherein the target region is a region whose disease index meets preset feature threshold conditions, the feature threshold conditions including: crack length greater than a preset length threshold; and / or corrosion area greater than a preset area threshold.

[0125] In one possible embodiment, controlling the underwater robot to perform fixed-distance observation of the object to be measured according to a preset navigation path includes: The instantaneous distance between the device and the surface of the object under test is measured in real time using a Doppler log. Calculate the difference between the instantaneous distance and the preset observation distance threshold, and adjust the lateral position of the underwater robot based on the difference to stabilize the instantaneous distance within the allowable fluctuation range of the observation distance threshold.

[0126] In one possible embodiment, after completing the observation of the object under test, constructing a three-dimensional reference model of the object under test based on each local three-dimensional model and the navigation pose data corresponding to each local three-dimensional model includes: Based on the navigation pose data corresponding to each of the local three-dimensional models, the spatial distribution information of each of the local three-dimensional models in the global coordinate system is determined. The spatial distribution information includes at least its projection contour on the horizontal reference plane and its depth information. Based on the spatial distribution of the projected contours of all local 3D models in the global coordinate system, a continuous 2D geometric shape is fitted as the inferred horizontal cross section of the object under test. Based on the inferred horizontal cross section and the depth information of each local 3D model, the 3D reference model is constructed along the vertical direction; or Based on the navigation pose data corresponding to each of the local three-dimensional models, the spatial distribution information of each of the local three-dimensional models in the global coordinate system is determined. The spatial distribution information includes at least its projection profile on the reference plane and its corresponding normal distance information. Based on the spatial distribution of all the projected profiles on the reference plane, a continuous two-dimensional geometry is fitted as the inferred two-dimensional cross-section of the object under test. Based on the inferred two-dimensional cross section and the normal distance information corresponding to each of the local three-dimensional models, the three-dimensional reference model is constructed along the normal of the reference datum plane.

[0127] In one embodiment, an underwater robot is provided, the internal structure of which can be shown in the following diagram. Figure 10 As shown, the underwater robot includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface allows the underwater robot to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a 3D modeling method for underwater targets.

[0128] In one embodiment, an underwater robot is proposed, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement... Figure 1 The method shown can be referred to for details. Figure 1 As shown, it will not be elaborated further here.

[0129] In one embodiment, a computer-readable storage medium is provided that stores a computer program, which is loaded and executed by a processor as described above. Figure 1The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figure 1 The specific details of the illustrated embodiments will not be elaborated here.

[0130] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or the underwater robot described above can be referred to the relevant descriptions on the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0131] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0132] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0133] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A 3D modeling method for underwater targets, characterized in that, The method includes: The underwater robot is controlled to perform fixed-distance observation of the object under test according to a preset navigation path, and two-dimensional observation images are acquired in real time through a camera device during the observation process. The two-dimensional observation image is identified in real time to confirm the existence of the target area; For the identified target area, a local three-dimensional model of the target area is constructed using a multi-view three-dimensional reconstruction algorithm based on the two-dimensional visual data of the target area. At the same time, the navigation pose data corresponding to the underwater robot when constructing each local three-dimensional model is recorded. After completing the observation of the object under test, a three-dimensional reference model of the object under test is constructed based on each of the local three-dimensional models and the navigation pose data corresponding to each of the local three-dimensional models. Based on the navigation pose data corresponding to each of the local 3D models, each of the local 3D models is filled into the 3D reference model to generate a 3D model of the object under test.

2. The 3D modeling method for underwater targets according to claim 1, characterized in that, The process of constructing a local 3D model of the target region using a multi-view 3D reconstruction algorithm based on the 2D visual data of the target region includes: Identify at least two projection markers within the two-dimensional observation image and establish a first reference dimension based on a preset fixed spacing between the corresponding transmitters; Two first feature points are selected within the target area, and the distance between the two first feature points is calculated based on the first reference size and used as the first size. The first dimension is mapped to three-dimensional space to form a second reference dimension; The local 3D model is constructed by combining the second reference dimension and the multi-view 3D reconstruction algorithm.

3. The 3D modeling method for underwater targets according to claim 2, characterized in that, The step of selecting two first feature points within the target area includes: If the target area is a crack area, select two endpoints on the geometric contour of the crack as the two first feature points; If the target area is a rusted area, then the two farthest corner points or the points with the greatest curvature on the closed contour of the rusted area are selected as the two first feature points.

4. The 3D modeling method for underwater targets according to claim 2, characterized in that, The construction of the local 3D model by combining the second reference size and the multi-view 3D reconstruction algorithm includes: Based on the second reference size, scale calibration and distortion correction are performed on the multi-view images acquired during the operation of the underwater robot to confirm a unified scale benchmark. The second feature points in each of the aforementioned viewpoint images are extracted, and the second feature points between different viewpoints are matched to establish a correspondence between viewpoints; The three-dimensional coordinates of the matched second feature point are calculated using triangulation to generate an initial sparse point cloud. Based on the actual spacing of the second reference size, the initial sparse point cloud is scaled and calibrated. The calibrated sparse point cloud is densified, and the local 3D model is generated by mesh reconstruction.

5. The 3D modeling method for underwater targets according to claim 2, characterized in that, The construction of the local 3D model by combining the second reference size and the multi-view 3D reconstruction algorithm includes: Use two of the first feature points within the target area as spatial fixed reference anchor points; Based on the pixel correspondence of two spatially fixed reference anchor points in multi-view images, and combined with multi-view geometric constraints, the epipolar geometric relationship of multiple view images is corrected to ensure the accuracy of spatial correspondence between different view images. Depth estimation is performed on each of the corrected viewpoint images to generate a multi-view depth map; Based on the second reference size, the calculated distance and actual spacing between the two spatial fixed reference anchor points in each depth map are compared, and the scale accuracy of each depth map is calibrated by a reverse iterative optimization algorithm. The calibrated multi-view depth maps are fused to generate a dense point cloud that conforms to the scale of the actual scene. After denoising the dense point cloud, the local 3D model is constructed.

6. The 3D modeling method for underwater targets according to claim 1, characterized in that, The real-time identification of the two-dimensional observation image to confirm the existence of the target area includes: At least one target region is identified from the two-dimensional observation image using a pre-trained target detection model; wherein the target region is a region whose disease index meets preset feature threshold conditions, the feature threshold conditions including: crack length greater than a preset length threshold; and / or corrosion area greater than a preset area threshold.

7. The 3D modeling method for underwater targets according to claim 1, characterized in that, The control of the underwater robot to perform fixed-distance observation of the object to be measured according to a preset navigation path includes: The instantaneous distance between the device and the surface of the object under test is measured in real time using a Doppler log. Calculate the difference between the instantaneous distance and the preset observation distance threshold, and adjust the lateral position of the underwater robot based on the difference to stabilize the instantaneous distance within the allowable fluctuation range of the observation distance threshold.

8. The 3D modeling method for underwater targets according to claim 1, characterized in that, After completing the observation of the object under test, a three-dimensional reference model of the object under test is constructed based on each of the local three-dimensional models and the navigation pose data corresponding to each local three-dimensional model, including: Based on the navigation pose data corresponding to each of the local three-dimensional models, the spatial distribution information of each of the local three-dimensional models in the global coordinate system is determined. The spatial distribution information includes at least its projection contour on the horizontal reference plane and its depth information. Based on the spatial distribution of the projected contours of all local 3D models in the global coordinate system, a continuous 2D geometric shape is fitted as the inferred horizontal cross section of the object under test. Based on the inferred horizontal cross section and the depth information of each local 3D model, the 3D reference model is constructed along the vertical direction; or Based on the navigation pose data corresponding to each of the local three-dimensional models, the spatial distribution information of each of the local three-dimensional models in the global coordinate system is determined. The spatial distribution information includes at least its projection profile on the reference plane and its corresponding normal distance information. Based on the spatial distribution of all the projected profiles on the reference plane, a continuous two-dimensional geometry is fitted as the inferred two-dimensional cross-section of the object under test. Based on the inferred two-dimensional cross section and the normal distance information corresponding to each of the local three-dimensional models, the three-dimensional reference model is constructed along the normal of the reference datum plane.

9. A 3D modeling device for underwater targets, characterized in that, The device includes: The acquisition module is used to control the underwater robot to perform fixed-distance observation of the object to be measured according to a preset navigation path. During the observation process, two-dimensional observation images are acquired in real time through a camera device. The identification module is used to identify the two-dimensional observation image in real time to confirm the existence of the target area; The construction module is used to construct a local three-dimensional model of the identified target area based on the two-dimensional visual data of the target area using a multi-view three-dimensional reconstruction algorithm, and at the same time record the navigation pose data of the underwater robot when constructing each local three-dimensional model; The construction module is further configured to, after completing the observation of the object under test, construct a three-dimensional reference model of the object under test based on each of the local three-dimensional models and the navigation pose data corresponding to each of the local three-dimensional models; The generation module is used to fill the three-dimensional reference model with the navigation pose data corresponding to each of the local three-dimensional models to generate a three-dimensional model of the object to be tested.

10. An underwater robot, characterized in that, The underwater robot includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the 3D modeling method for the underwater target as described in any one of claims 1 to 8.

Images