An underwater scene three-dimensional reconstruction method based on coded ruler

Abstract

The application relates to a kind of underwater scene three-dimensional reconstruction methods based on coded ruler, belong to three-dimensional reconstruction technical field, including the coded ruler, the three-dimensional space coordinates of the coded point on the coded ruler of the underwater target to be measured fixed are measured, the two-dimensional pixel coordinates of the coded point in the environment image collected are extracted, and it is paired with three-dimensional space coordinates, by the underwater image conversion model established, two-dimensional pixel coordinates are converted into equivalent air pixel coordinates, and the matching point pair is solved by geometric constraint, three-dimensional local coordinates are obtained, initial camera pose is calculated, initial camera pose is optimized, and the optimized camera pose is obtained, after the surface line structure light image of the underwater target to be measured collected is extracted laser center line, three-dimensional reconstruction is carried out, and local point cloud is obtained, based on the optimized camera pose, local point cloud is fused and reconstructed, and three-dimensional surface model is output, low cost, high robustness underwater mobile three-dimensional reconstruction is realized.

Description

Technical Field

[0001] This invention relates to the field of 3D reconstruction technology, and more specifically to a method for 3D reconstruction of underwater scenes based on a coding scale. Background Technology

[0002] With the development of underwater exploration, marine engineering, and industrial inspection, there is an urgent need for high-precision 3D reconstruction of underwater targets. Traditional fixed binocular structured light systems are limited by the camera's field of view, making it difficult to complete the full scanning of large underwater structures or wide-area scenes. Mounting 3D sensors on underwater vehicles or robotic arms for mobile scanning and reconstruction has become the mainstream research direction.

[0003] Currently, existing underwater mobile 3D reconstruction methods mainly include three technical approaches. The first approach relies on acoustic positioning sensors or inertial sensors, such as inertial navigation systems, Doppler logs, or ultra-short baseline positioning systems, to acquire motion trajectories and stitch local point clouds from each frame to the global coordinate system. The second approach introduces visual odometry or simultaneous localization and mapping algorithms to perform inter-frame pose tracking by extracting underwater natural feature points. The third approach directly utilizes local point clouds and performs rigid registration between point clouds through iterative nearest-point algorithms and their variants.

[0004] In the aforementioned existing technologies, acoustic positioning systems or inertial sensors have high hardware costs and drift errors that accumulate over time, which can cause severe geometric distortion of the 3D model during long-distance scanning; visual positioning methods based on natural feature points are prone to failure in underwater scattering environments and weak texture environments; point cloud registration methods also suffer from accumulated registration errors frame by frame due to the sparseness of single-frame point clouds and the lack of global reference constraints, which can eventually cause the model to fail to close at the beginning and end or to bend as a whole. Summary of the Invention

[0005] To address the shortcomings of related technologies, the present invention aims to provide a method for underwater scene 3D reconstruction based on an coded scale, thereby solving the technical problems of existing mobile underwater 3D reconstruction technologies that rely on expensive sensors or are susceptible to underwater environmental interference, and that cumulative drift errors lead to geometric distortion of the model.

[0006] This invention provides a method for 3D reconstruction of underwater scenes based on an coded scale, comprising: Ruler setup steps: Fix the coded ruler with coded points printed on it to the underwater target to be measured, and measure the three-dimensional spatial coordinates of all coded points on the coded ruler; Point set establishment steps: Acquire environmental images containing the coded scale and the underwater target to be measured through a camera, extract the two-dimensional pixel coordinates of the coded points in the environmental images, and pair the two-dimensional pixel coordinates and the three-dimensional spatial coordinates of the same coded point to obtain a matching point set; Refraction compensation step: Obtain the optical parameters of the camera, establish an underwater image conversion model based on the optical parameters, and convert the two-dimensional pixel coordinates in the matching point set into equivalent air pixel coordinates through the underwater image conversion model to obtain an equivalent matching point set; Pose estimation steps: Select matching point pairs from the equivalent matching point set, solve the geometric constraints for each selected matching point pair to obtain the three-dimensional local coordinates of the encoded points in each matching point pair, calculate the initial camera pose based on the three-dimensional spatial coordinates and the three-dimensional local coordinates, and optimize the initial camera pose to obtain the optimized camera pose; 3D Reconstruction Steps: Acquire a line structured light image of the surface of the underwater target to be tested. After refraction compensation of the line structured light image using the underwater image conversion model, extract the laser center line. Perform 3D reconstruction on the laser center line to obtain a local point cloud. Based on the optimized camera pose, fuse and reconstruct the local point cloud to output a 3D surface model.

[0007] By pre-measuring the three-dimensional spatial coordinates of all coding points on the coding scale, the two-dimensional pixel coordinates of the coding points are extracted from the image and paired with the three-dimensional spatial coordinates, providing a known absolute reference for subsequent positioning; By establishing an underwater image conversion model to compensate for refraction distortion, distorted pixels are converted into equivalent air pixels, avoiding the divergence problem of the traditional PnP algorithm due to refraction nonlinearity. This ensures that the two-dimensional and three-dimensional matching relationship strictly meets the perspective projection constraints, providing an accurate data foundation for pose solving. Furthermore, local point clouds were obtained through 3D reconstruction of line structured light images, and fusion reconstruction was performed based on optimized camera pose, achieving high-precision underwater 3D reconstruction without relying on inertial navigation or acoustic positioning equipment.

[0008] In some embodiments of the present invention, the scale arrangement step specifically includes: A world coordinate system is established with the center of the encoding point at a preset position on the encoding scale as the origin, the length direction of the encoding scale as the X-axis, the width direction of the encoding scale as the Y-axis, and the direction perpendicular to the plane of the encoding scale as the Z-axis. The three-dimensional spatial coordinates of all coding points in the world coordinate system are measured using a coordinate measuring machine, and the code value and its three-dimensional spatial coordinates corresponding to each coding point are stored in a lookup table.

[0009] By establishing a world coordinate system with the center of the encoding point at a preset position on the encoding scale as the origin, and using a coordinate measuring machine to accurately determine the three-dimensional spatial coordinates of all encoding points in advance, and storing the code value of each encoding point and its coordinates in a lookup table, a one-to-one correspondence between any encoding point and its absolute spatial position is achieved. This allows the three-dimensional coordinates of the corresponding point to be quickly obtained by decoding during subsequent image processing, which greatly improves positioning efficiency and data accuracy and provides a reliable global reference for eliminating cumulative drift.

[0010] In some embodiments of the present invention, the point set establishment step specifically includes: Extract the center position of the encoded points in the environmental image to obtain the two-dimensional pixel coordinates of each encoded point on the environmental image; The extracted encoding points are decoded to obtain the code value corresponding to the encoding point. Based on the code value, the corresponding three-dimensional spatial coordinates are retrieved from the lookup table. The matching point set is established based on the three-dimensional spatial coordinates and the two-dimensional pixel coordinates of the same encoding point.

[0011] By extracting the center position of the coded points in the environmental image to obtain the two-dimensional pixel coordinates, decoding the coded points to obtain their code values, and then retrieving the corresponding three-dimensional spatial coordinates in the lookup table, the two-dimensional coordinates and three-dimensional coordinates of the same coded point are paired to establish a matching point set. This achieves an accurate correspondence between underwater image feature points and their real spatial positions, providing an accurate data source for subsequent pose estimation, avoiding pose deviations caused by feature point matching errors, and improving the positioning reliability of the system.

[0012] In some embodiments of the present invention, the refractive compensation step specifically includes: The equivalent focal length of the camera, the unit normal vector of the waterproof glass plane outside the camera, and the distance from the optical center of the camera to the waterproof glass plane are obtained as the optical parameters. Based on the optical parameters, calculate the coordinates of the intersection point between the light beam projected from the pixel corresponding to the two-dimensional pixel coordinates and the plane of the waterproof cover glass. Based on the intersection coordinates, the mapping relationship between the two-dimensional pixel coordinates and the equivalent air pixel coordinates is determined by the law of refraction, thus obtaining the underwater image conversion model.

[0013] By obtaining the equivalent focal length of the camera, the unit normal vector of the waterproof cover glass plane, and the distance from the optical center to the glass plane as optical parameters, the coordinates of the intersection point between the back-projected beam and the glass plane are calculated. Then, the mapping relationship between pixel coordinates and equivalent air pixel coordinates is established through the law of refraction, so that the underwater curved optical path is accurately converted into an equivalent straight optical path. This eliminates the destructive effect of underwater multi-layer medium refraction on imaging geometry and provides accurate input data that conforms to the linear perspective projection constraints for subsequent pose calculation.

[0014] In some embodiments of the present invention, the refractive compensation step further includes: Based on the aforementioned optical parameters, the refractive index scaling factor is derived using the law of refraction. Obtain the approximate depth of the encoded points in the matching point set. Based on the refractive index scale factor and the approximate depth, convert the two-dimensional pixel coordinates of each encoded point in the matching point set into the corresponding equivalent air pixel coordinates to obtain the equivalent matching point set.

[0015] By deriving the refractive index scaling factor based on the law of refraction using optical parameters and combining it with the approximate depth of the coding point, the two-dimensional pixel coordinates are converted into equivalent air pixel coordinates, thus achieving accurate numerical compensation for underwater refraction distortion. This ensures that the converted pixel coordinates strictly satisfy the perspective projection model in air, avoids interference from refractive nonlinearity on PnP pose solving, and guarantees the geometric consistency between the two-dimensional observation values ​​and the three-dimensional spatial coordinates of the coding point.

[0016] In some embodiments of the present invention, the pose estimation step specifically includes: Based on the equivalent air pixel coordinates, calculate the cosine value of the angle between the optical center of the camera and the line of sight to the coding point of each matching point pair; Based on the three-dimensional spatial coordinates and the cosine value of the included angle, a geometric constraint equation is constructed using the cosine theorem. The geometric constraint equation is then solved to obtain the three-dimensional local coordinates of the coding points in each matching point pair. Based on the three-dimensional spatial coordinates and three-dimensional local coordinates of the coded points in each matching point pair, the rotation matrix and translation vector are calculated through singular value decomposition to obtain the initial camera pose.

[0017] The cosine of the line-of-sight angle is calculated using equivalent air pixel coordinates, and geometric constraint equations are constructed using the cosine theorem in combination with 3D spatial coordinates. The 3D local coordinates of the encoded point in the camera coordinate system are then obtained. Furthermore, the rotation matrix and translation vector are calculated between the world coordinates and the camera coordinates through singular value decomposition. This achieves accurate solution from 2D observation to 3D pose, providing independent camera pose estimation for each frame and blocking the error propagation chain at its source.

[0018] In some embodiments of the present invention, the pose estimation step further includes: The three-dimensional spatial coordinates of the encoded points in each of the matching point pairs are reprojected to obtain the reprojected pixels. The deviation between the reprojected pixel and the corresponding equivalent air pixel coordinates is calculated to obtain the reprojection error; With the goal of minimizing the reprojection error, the initial camera pose is iteratively optimized to obtain the optimized camera pose.

[0019] By reprojecting the three-dimensional spatial coordinates to obtain the reprojected pixels, the deviation between the reprojected pixels and the equivalent air pixel coordinates is calculated as the reprojection error. The reprojection error of all matching point pairs is minimized through iterative optimization, so that the camera pose parameters reach the optimum in the global range. This effectively suppresses the noise influence introduced in the feature extraction and polygon solving process, and greatly improves the accuracy and numerical stability of pose calculation.

[0020] In some embodiments of the present invention, the three-dimensional reconstruction step specifically includes: Acquire a line structured light image of the surface of the underwater target to be tested; The center line of the laser bright stripe in the line structured light image is extracted to obtain an initial center line coordinate sequence. The initial center line coordinate sequence is then smoothed to obtain the laser center line.

[0021] By acquiring line structured light images and extracting the center line of the laser bright stripe, and then smoothing the initial center line coordinate sequence, high-frequency noise and pixel jumps caused by underwater suspended particles scattering are effectively filtered out. While preserving the original curved shape of the light stripe, the jagged burrs are eliminated, providing smooth and accurate laser center line features for subsequent stereo matching, and ensuring the purity and geometric accuracy of the underlying data for 3D reconstruction.

[0022] In some embodiments of the present invention, the three-dimensional reconstruction step further includes: The laser center lines are stereo matched along the epipolar direction to determine the matching pixel pairs between each laser center line; Based on the positional differences between pixels in the pixel pair, the three-dimensional spatial coordinates corresponding to each pixel are calculated to obtain the local point cloud.

[0023] By performing stereo matching along the epipolar direction of the laser centerline, corresponding pixel pairs in the optical structure image are determined. Based on the positional differences of the pixel pairs, the three-dimensional spatial coordinates are calculated through triangulation, thus realizing the accurate reconstruction from two-dimensional laser stripes to three-dimensional spatial points. This overcomes the problem of sparse single-frame point clouds in underwater scattering environments and provides high-precision local three-dimensional data for subsequent multi-frame point cloud fusion.

[0024] In some embodiments of the present invention, the three-dimensional reconstruction step further includes: Based on the optimized camera pose, the local point cloud is transformed to obtain the global point cloud; The global point cloud is accumulated to obtain a complete point cloud. After filtering and denoising the complete point cloud, a continuous triangular mesh is generated by a surface reconstruction algorithm to obtain the three-dimensional surface model.

[0025] By optimizing the camera pose, the local point clouds of each frame are transformed to the same coordinate system for accumulation. The complete point cloud is then filtered, denoised, and reconstructed. This allows each frame of point cloud to be stitched together directly based on a global absolute reference, fundamentally eliminating the inherent defect of error accumulation with distance in traditional inter-frame recursive methods. Ultimately, a high-precision continuous 3D surface model without geometric distortion and faults is generated. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart of an underwater scene 3D reconstruction method based on an encoding scale provided in an embodiment of the present invention; Figure 2 A schematic diagram of an encoding scale provided in an embodiment of the present invention; Figure 3 A schematic diagram of an underwater image conversion model provided in an embodiment of the present invention; Figure 4 A schematic diagram of the initial rigid body transformation for Singular Value Decomposition (SVD) provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a light stripe center filtering process provided in an embodiment of the present invention; Figure 6 for Figure 5 Enlarged view of a portion of point A in the middle. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0029] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0030] With the rapid development of underwater exploration, marine engineering and industrial inspection, there is an urgent need for high-precision three-dimensional reconstruction of underwater targets. Traditional underwater three-dimensional measurement usually adopts a fixed binocular structured light system, that is, the camera and the target remain relatively stationary during the scanning process. However, fixed systems are limited by the effective field of view of the camera, making it difficult to complete a full scan of large underwater structures or wide-area underwater scenes. In order to overcome the field of view limitation, mounting 3D sensors on underwater vehicles or robotic arms for mobile scanning and reconstruction has become the mainstream research direction. In mobile 3D reconstruction, the core technical challenge lies in the high-precision stitching of multi-frame point clouds and global pose localization. Currently, underwater robots typically rely on inertial navigation systems, Doppler logs, or ultra-short baseline positioning systems to obtain motion trajectories, and then stitch the local point clouds of each frame into the global coordinate system. To reduce costs, some studies introduce visual odometry or simultaneous localization and mapping algorithms to achieve pose estimation of underwater cameras. In addition, there are methods that do not rely on external positioning equipment and directly utilize local point clouds to perform rigid registration between point clouds through iterative nearest point and its variant algorithms. However, the aforementioned existing technologies all have obvious defects and shortcomings; among them, the hardware cost of high-precision inertial navigation systems or Doppler logs is extremely high, making it difficult to popularize them in micro underwater robots or low-cost operations. Inertial sensors calculate pose by integrating acceleration and angular velocity, which inevitably leads to drift errors that accumulate infinitely over time. In long-distance or long-term scanning operations, this can cause severe geometric distortion and discontinuity in the stitched 3D model, making it unable to meet the requirements of high-precision industrial measurement. Visual odometry or simultaneous localization and mapping algorithms based on natural feature point matching are extremely vulnerable in underwater environments. The underwater environment suffers from severe scattering by suspended particles and light attenuation, resulting in extremely low signal-to-noise ratios in images, making it difficult to extract stable, high-quality natural feature points. Typical industrial measurement targets, such as underwater pipelines or nuclear fuel pool walls, often lack rich texture information on their surfaces, making it easy for feature-matching-based inter-frame pose tracking to get stuck in local minima or lose track of the target altogether. Furthermore, visual odometry or simultaneous localization and mapping algorithms are essentially still calculating the relative pose between frames and then recursively obtaining the global pose. This recursive process also produces accumulated errors that cannot be completely eliminated. The point cloud registration method based on iterative nearest point only obtains sparse linear point clouds in a single frame of line structured light scanning, lacking sufficient geometric features and overlap rate for the algorithm to perform robust matching. Even if multiple frames are accumulated to form a planar point cloud and then registered, the registration error will still accumulate frame by frame during long-distance scanning due to the lack of global reference constraints, eventually causing the model to fail to close at the beginning and end or to bend as a whole.

[0031] In summary, existing underwater mobile 3D reconstruction methods, whether relying on hardware-based inertial navigation systems or software-based visual positioning or point cloud registration, cannot avoid the underlying logical flaw of error accumulation caused by inter-frame extrapolation, and have extremely poor robustness in low-texture, high-scattering underwater environments. Therefore, there is an urgent need in this field for a low-cost, highly robust underwater mobile 3D reconstruction method that can fundamentally eliminate accumulated drift errors in order to meet the high-precision measurement requirements of complex underwater large-scale targets.

[0032] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0033] The technical solution of the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.

[0034] like Figure 1 As shown, this invention provides a method for three-dimensional reconstruction of underwater scenes based on an coded scale, comprising: Step S1: Fix the coded scale with coded points on it to the underwater target to be measured, and measure the three-dimensional spatial coordinates of all coded points on the coded scale. In some embodiments, the scale arrangement step S1 specifically includes: A world coordinate system is established with the center of the coding point at the preset position on the coding scale as the origin, the length direction of the coding scale as the X-axis, the width direction of the coding scale as the Y-axis, and the direction perpendicular to the plane of the coding scale as the Z-axis. The coordinates of all coding points in the world coordinate system are measured using a coordinate measuring machine, and the code value and its three-dimensional coordinates corresponding to each coding point are stored in a lookup table.

[0035] By establishing a world coordinate system with the center of the encoding point at a preset position on the encoding scale as the origin, and using a coordinate measuring machine to accurately determine the three-dimensional spatial coordinates of all encoding points in advance, and storing the code value of each encoding point and its coordinates in a lookup table, a one-to-one correspondence between any encoding point and its absolute spatial position is achieved. This allows the three-dimensional coordinates of the corresponding point to be quickly obtained by decoding during subsequent image processing, which greatly improves positioning efficiency and data accuracy and provides a reliable global reference for eliminating cumulative drift.

[0036] Furthermore, such as Figure 2 As shown, in order to establish the absolute spatial reference required for subsequent three-dimensional reconstruction, a coding scale is fixed near the underwater target to be measured. The coding scale is a planar carbon fiber target with high-precision ring coding points printed on it. Each coding point on the target is specially designed, and its ring has a unique black and white arrangement pattern, corresponding to a unique code value. This code value is equivalent to the identification number of the coding point. The center of the coding point at a preset position on the target is selected as the origin of the coordinate system. With the length direction of the target as the X-axis With the width of the target as the Y-axis The Z-axis is the direction perpendicular to the target plane. Establish a global world coordinate system ; A set of precise three-dimensional spatial coordinates of all coded points in the world coordinate system is pre-measured using a high-precision coordinate measuring machine. The code value of each coding point and its three-dimensional spatial coordinates are stored in a preset lookup table, so that the lookup table stores the data relationship between all code values ​​and their absolute three-dimensional coordinates in the world coordinate system.

[0037] Point set establishment step S2: Acquire environmental images containing coded scales and underwater targets to be measured through a camera, extract the two-dimensional pixel coordinates of coded points in the environmental images, and pair the two-dimensional pixel coordinates and the three-dimensional spatial coordinates of the same coded point to obtain a matching point set; In some embodiments, the point set establishment step S2 specifically includes: Extract the center position of the encoded points in the environment image to obtain the two-dimensional pixel coordinates of each encoded point in the environment image; The extracted encoded points are decoded to obtain the code values ​​corresponding to the encoded points. Based on the code values, the corresponding three-dimensional spatial coordinates are retrieved from the lookup table. A matching point set is established based on the three-dimensional spatial coordinates and two-dimensional pixel coordinates of the same encoded point.

[0038] By extracting the center position of the coded points in the environmental image to obtain the two-dimensional pixel coordinates, decoding the coded points to obtain their code values, and then retrieving the corresponding three-dimensional spatial coordinates in the lookup table, the two-dimensional coordinates and three-dimensional coordinates of the same coded point are paired to establish a matching point set. This achieves an accurate correspondence between underwater image feature points and their real spatial positions, providing an accurate data source for subsequent pose estimation, avoiding pose deviations caused by feature point matching errors, and improving the positioning reliability of the system.

[0039] Furthermore, during the first phase of mobile scanning... At frame time, the left camera of the underwater binocular system is used as the main camera to acquire an environmental image that simultaneously contains the coded scale and the underwater target to be measured. In this environmental image, each coded point on the coded scale appears as an ellipse due to the perspective projection effect. The system uses ellipse fitting and edge detection algorithms to process environmental images. First, the image is converted to grayscale, and then the inner and outer edge pixels of the coded point ring are extracted by the edge detection operator to obtain the image coordinate set of the edge points. Then, the system uses the least squares method to fit the extracted edge point set to an ellipse and establish an ellipse equation. The ellipse equation describes the shape and position of the coded point ring in the image. Finally, the equation of the ellipse is solved to calculate the image coordinates of the ellipse's center. These center coordinates are the two-dimensional pixel coordinates of the encoded point. ; Through the above ellipse fitting process, even if the coded point is distorted by perspective due to the shooting angle, the system can still accurately fit the ellipse based on the edge point of the ring and locate its center, ensuring the sub-pixel accuracy of the coded point center positioning, and providing high-precision image coordinates for subsequent two-dimensional to three-dimensional matching. At the same time, the system decodes the black and white pattern within the same coding point ring to obtain the unique code value of the coding point. After obtaining the code value, the system retrieves the code value in a pre-established lookup table. Since the lookup table has pre-stored the data relationship between all code values ​​and their absolute three-dimensional coordinates in the world coordinate system, the system can quickly find the three-dimensional spatial coordinates corresponding to the coding point. Subsequently, the system pairs the two-dimensional pixel coordinates of the same encoding point with the three-dimensional spatial coordinates to form a set of two-dimensional to three-dimensional matching point pairs; By performing the extraction, decoding, retrieval, and pairing operations described above on all identifiable coded points in the environmental image, the first... Under the frame Set of 2D to 3D matching points This provides basic data for subsequent camera pose calculation; Through the above process, the three-dimensional coordinates of the encoded point can be obtained quickly by decoding alone, without the need for real-time measurement or reliance on other positioning devices, which greatly improves positioning efficiency and data accuracy, and provides a reliable global reference for eliminating inter-frame cumulative drift.

[0040] Refraction compensation step S3: Obtain the camera's optical parameters, establish an underwater image conversion model based on the optical parameters, and convert the two-dimensional pixel coordinates in the matching point set into equivalent air pixel coordinates through the underwater image conversion model to obtain the equivalent matching point set; In some embodiments, the refraction compensation step S3 specifically includes: The equivalent focal length of the camera, the unit normal vector of the waterproof glass plane set outside the camera, and the distance from the optical center of the camera to the waterproof glass plane are obtained as optical parameters. Based on optical parameters, calculate the coordinates of the intersection point between the beam projected from the pixel corresponding to the two-dimensional pixel coordinates and the plane of the waterproof cover glass. Based on the intersection coordinates, the mapping relationship between the two-dimensional pixel coordinates and the equivalent air pixel coordinates is determined by the law of refraction, thus obtaining the underwater image conversion model.

[0041] By obtaining the equivalent focal length of the camera, the unit normal vector of the waterproof cover glass plane, and the distance from the optical center to the glass plane as optical parameters, the coordinates of the intersection point between the back-projected beam and the glass plane are calculated. Then, the mapping relationship between pixel coordinates and equivalent air pixel coordinates is established through the law of refraction, so that the underwater curved optical path is accurately converted into an equivalent straight optical path. This eliminates the destructive effect of underwater multi-layer medium refraction on imaging geometry and provides accurate input data that conforms to the linear perspective projection constraints for subsequent pose calculation.

[0042] Furthermore, since the camera of the underwater binocular system is sealed in a waterproof cover with a flat glass plate, the light needs to pass through three media in sequence—water, glass, and air—during the actual imaging process. Nonlinear refraction occurs at the interface between these media, causing the light to be unable to propagate in a straight line. If the two-dimensional pixel coordinates directly acquired by the camera are substituted into the PnP algorithm based on linear perspective projection for pose solving, the geometric inconsistency between the two, since the actual light path is a curved light path while the algorithm assumes that the light path is a straight propagation, will cause the pose solution to diverge or produce significant errors. Therefore, the curved light path that actually occurs underwater must be mapped to an equivalent straight light path in air so that the mapped pixel coordinates satisfy the linear perspective projection constraint.

[0043] First, define the image coordinate system. The image coordinate system is a two-dimensional coordinate system on the camera imaging plane, used to describe the specific pixel position of an object point in three-dimensional space on the image after it is imaged by the camera. Taking the top-left corner of the image plane as the origin, the horizontal direction to the right is... The positive direction of the axis, the vertical downward direction is In the positive direction of the axis, any pixel in the image can be uniquely determined by a set of two-dimensional coordinates, where the horizontal coordinate represents the column position of the pixel in the image, and the vertical coordinate represents the row position of the pixel in the image.

[0044] The "image" in the image coordinate system does not refer to a specific image, but rather to the imaging plane itself where the camera sensor is located. That is, any image captured by the camera uses this coordinate system as a unified reference for pixel position. The two-dimensional pixel coordinates of coded points in environmental images and the pixel coordinates of the laser centerline in structured light images are both extracted within this image coordinate system. Define the camera coordinate system. The camera coordinate system is a three-dimensional Cartesian coordinate system fixed on the camera, used to describe the position and orientation of an object in three-dimensional space relative to the camera.

[0045] With the optical center of the camera as the origin of the coordinate system, shaft and The axes are parallel to the image coordinate system. shaft and axial direction, i.e. The axis is horizontal to the right. The axis is vertically downward. The axis is perpendicular to the imaging plane and points towards the scene along the camera's optical axis, that is, towards the front of the camera; Any point in the camera coordinate system can be represented by three-dimensional coordinates, where the coordinates along the optical axis represent the depth distance from the point to the camera's optical center. Points in the camera coordinate system are mapped to the image coordinate system through perspective projection, and the mapping relationship is described by the camera intrinsic parameter matrix.

[0046] Based on the above analysis, the system obtains the equivalent focal length of the camera, which reflects the imaging geometry of the camera in the air. Simultaneously, obtain the unit normal vector of the glass plane of the waterproof cover set outside the camera, which describes the spatial orientation of the glass plane in the camera coordinate system; During underwater imaging, light rays are refracted at the flat surface of the waterproof glass cover, which is called the refractive plane. The system also obtains the distance from the camera's optical center to the plane of the waterproof housing glass, which determines the relationship between the optical center and the refractive plane. The relative positional relationship between them; The equivalent focal length, unit normal vector, and distance mentioned above are used as optical parameters.

[0047] like Figure 3 As shown, the camera coordinate system is defined as follows: The image coordinate system is ; For the refracting plane The unit normal vector, where These are the components of the normal vector along the three axes of the camera coordinate system, and this vector describes the spatial orientation of the glass plane in the camera coordinate system. For camera optical center The distance to the plane of refraction; For camera optical center And parallel to the plane of refraction The plane; A point in space The coordinates in the camera coordinate system are represented as follows: Its imaging point in the presence of water is The imaging point in the absence of water is .

[0048] Based on the acquired optical parameters, for a pixel point corresponding to a certain two-dimensional pixel coordinate in the matching point set... The system calculates the intersection point of the beam projected from that pixel and the plane of the waterproof cover glass. coordinates in the camera coordinate system The calculation formula is as follows:

[0049]

[0050]

[0051] in, The intersection point represents the two-dimensional pixel coordinates of the pixel in the image coordinate system; this intersection point is the actual incident point of the light on the glass plane, and is also a key parameter for subsequent refraction calculations.

[0052] in, For plane With past The intersection of the normals to the plane of refraction of a point; For plane With refracted light The intersection of the reverse extensions; Based on the above intersection coordinates, the mapping relationship between the two-dimensional pixel coordinates and the equivalent air pixel coordinates is determined by the law of refraction, and an underwater image conversion model is obtained. According to the optical path tracing method, the light rays projected from the underwater imaging point intersect the glass plane at the incident intersection point, which is the actual incident point of the light rays at the interface between the underwater medium and the glass. Given the coordinates of the intersection point and the normal vector of the glass plane, the system calculates the angle of incidence and the angle of refraction of the light at the incident intersection point according to Snell's law, and determines the direction of refraction of the light after it enters the glass plane from the underwater medium; Then, by continuing to trace the refraction process of light at the glass-air interface, the equivalent straight light path when the light reaches the camera sensor is finally obtained. The intersection of this equivalent straight light path and the image plane is the equivalent air imaging point. Using the above method, a mapping relationship between any underwater imaging point and its equivalent air imaging point can be established, thereby obtaining an underwater image conversion model. This model takes the two-dimensional pixel coordinates of each coded point in the matching point set as input, and outputs the corresponding equivalent air pixel coordinates after mapping calculation, updating the matching point set to an equivalent matching point set that satisfies the linear perspective projection model.

[0053] In some embodiments, the refraction compensation step S3 further includes: Based on optical parameters, the refractive index scaling factor is derived using the law of refraction. Obtain the approximate depth of the encoded points in the matching point set. Based on the refractive index scale factor and the approximate depth, convert the two-dimensional pixel coordinates of each encoded point in the matching point set into the corresponding equivalent air pixel coordinates to obtain the equivalent matching point set.

[0054] By deriving the refractive index scaling factor based on the law of refraction using optical parameters and combining it with the approximate depth of the coding point, the two-dimensional pixel coordinates are converted into equivalent air pixel coordinates, thus achieving accurate numerical compensation for underwater refraction distortion. This ensures that the converted pixel coordinates strictly satisfy the perspective projection model in air, avoids interference from refractive nonlinearity on PnP pose solving, and guarantees the geometric consistency between the two-dimensional observation values ​​and the three-dimensional spatial coordinates of the coding point.

[0055] Furthermore, based on the ray tracing method to analyze the propagation process of the back-projected rays from the object point, and according to Snell's law of refraction, it can be known that... ,in and These are the refractive indices of air and water, respectively. The angle of refraction of light on the air side. The angle of refraction of light on the water side; And thus the angle of refraction It can be represented as:

[0056] in From the formula Determine the refractive index scaling factor, and then derive it. ; Obtain the depth value of the encoded point in the matching point set in the camera coordinate system, that is, the distance from the point to the camera optical center along the optical axis, to obtain the approximate depth. Based on the coordinates of the intersection point of the light ray and the glass plane Approximate depth of coding points Combined with refractive index scaling factor Calculate the equivalent air pixel coordinates after eliminating refractive distortion. :

[0057]

[0058] in, As an intermediate variable, .

[0059] Through the above calculations, for each coded point in the matching point set, the system calculates the corresponding equivalent air pixel coordinates based on its two-dimensional pixel coordinates, and updates the matching point set to an equivalent matching point set that strictly satisfies the linear perspective projection model. Thus, the underwater curved optical path is accurately converted into an equivalent straight optical path, eliminating the destructive effect of underwater multi-layer medium refraction on imaging geometry, and providing accurate input data that conforms to the linear perspective projection constraints for subsequent pose solving.

[0060] Pose estimation step S4: Select matching point pairs from the equivalent matching point set, solve the geometric constraints for each selected matching point pair, obtain the three-dimensional local coordinates of the encoded points in each matching point pair, calculate the initial camera pose based on the three-dimensional spatial coordinates and the three-dimensional local coordinates, optimize the initial camera pose, and obtain the optimized camera pose. In some embodiments, pose estimation step S4 specifically includes: Based on the equivalent air pixel coordinates, calculate the cosine value of the angle between the camera's optical center and the line of sight to each matching point-to-center coding point; Based on the three-dimensional spatial coordinates and the cosine value of the included angle, a geometric constraint equation is constructed using the cosine theorem. The geometric constraint equation is then solved to obtain the three-dimensional local coordinates of the centering coding points of each matching point pair. Based on the three-dimensional spatial coordinates and three-dimensional local coordinates of the coded points in each matching point pair, the rotation matrix and translation vector are calculated through singular value decomposition to obtain the initial camera pose.

[0061] The cosine of the line-of-sight angle is calculated using equivalent air pixel coordinates, and geometric constraint equations are constructed using the cosine theorem in combination with 3D spatial coordinates. The 3D local coordinates of the encoded point in the camera coordinate system are then obtained. Furthermore, the rotation matrix and translation vector are calculated between the world coordinates and the camera coordinates through singular value decomposition. This achieves accurate solution from 2D observation to 3D pose, providing independent camera pose estimation for each frame and blocking the error propagation chain at its source.

[0062] Furthermore, in order to solve the absolute pose of the camera relative to the global scale for each frame, the equivalent air pixel coordinates calculated above are incorporated into the RPnP framework. From the equivalent matching point set Three pairs of matched points are randomly selected to form a P3P subset. Let the pairwise distances between the three selected coded points in the camera coordinate system be... The known physical distance between any two of them in the world coordinate system is... ; Using the equivalent air pixel coordinates of these three encoding points, calculate the cosine of the angle between the camera optical center and the line of sight to each encoding point. , , Based on the law of cosines, the following system of nonlinear equations is constructed:

[0063] By transforming the above nonlinear equations into univariate quartic polynomial equations and solving for the real roots, the three-dimensional local coordinates of the three coding points in the camera coordinate system can be initially obtained. Repeat the above operation for each pair of matching points in the equivalent matching point set to obtain the three-dimensional local coordinates of the coded points in each matching point pair in the camera coordinate system. ; like Figure 4 As shown, the left camera coordinate system is defined. The left camera coordinate system has its origin at the optical center of the left camera. , shaft and The axes are parallel to the horizontal and vertical directions of the image plane, respectively. The axis points along the optical axis of the left camera into the scene's three-dimensional Cartesian coordinate system. Define the right camera coordinate system The right camera coordinate system has its origin at the optical center of the right camera. , shaft and The axes are parallel to the horizontal and vertical directions of the image plane, respectively. The axis points along the optical axis of the right camera into a three-dimensional Cartesian coordinate system that points to the scene. Define the first rotation matrix from the left camera coordinate system to the world coordinate system. and the first translation matrix First rotation matrix This describes the relative orientation relationship between two coordinate systems; the first translation matrix... It describes the position offset of the optical center of the left camera relative to the origin of the world coordinate system; Let be a coded point in the world coordinate system, and let its coordinates be the coordinates of the image point on the image. Its coordinate system in the left camera coordinate system The coordinates below are ; For any 3D point in the left camera coordinate system Its coordinates in the world coordinate system Through formula Calculated; Define the second rotation matrix from the left camera coordinate system to the right camera coordinate system. Second translation matrix Second rotation matrix The second translation matrix describes the orientation relationship of the right camera relative to the left camera. It describes the positional offset of the optical center of the right camera relative to the optical center of the left camera; For any 3D point in the left camera coordinate system Its coordinates in the right camera coordinate system Through formula Calculated; The absolute pose of the camera is solved by the initial rigid body transformation of the singular value decomposition (SVD). The initial rotation matrix and translation vector of the camera are calculated by the singular value decomposition algorithm based on the obtained three-dimensional local coordinates of all encoded points in the camera coordinate system and the absolute three-dimensional coordinates of each encoded point in the world coordinate system. Specifically, it is known The set of three-dimensional spatial coordinates of each encoded point in the world coordinate system and the set of corresponding 3D local coordinates in the camera coordinate system. The objective is to find the optimal initial rotation matrix. Translation vector ; First, calculate the centroids of the two sets of three-dimensional coordinates:

[0064]

[0065] in, It is the centroid of the set of three-dimensional spatial coordinates in the world coordinate system; Let be the centroid of the set of three-dimensional local coordinates in the camera coordinate system.

[0066] Decentralize the two sets of 3D coordinates:

[0067]

[0068] in, and These are the decentralized 3D spatial coordinates and 3D local coordinates, respectively.

[0069] Construct using a decentralized set of 3D coordinates Covariance matrix:

[0070] For covariance matrix Singular value decomposition yields three matrices. , , Its expression is:

[0071] in, and for orthogonal matrix, It is a diagonal matrix containing singular values; Based on the singular value decomposition results, calculate the closed-form solution of the initial rotation matrix:

[0072] If the calculated rotation matrix The determinant of the matrix is ​​less than zero, that is... This indicates that the matrix includes reflection transformation, which does not conform to the physical laws of rigid body rotation; At this point, the matrix needs to be... The last column is inverted for correction. The correction formula is:

[0073] The obtained rotation matrix Substituting back to the centroid relation, we obtain the translation vector:

[0074] Thus, the initial rotation matrix with high precision was obtained using the singular value decomposition method. Translation vector Combined with the initial rotation matrix Translation vector Obtain the initial camera pose ; The cosine of the line-of-sight angle is calculated using equivalent air pixel coordinates. Combined with 3D spatial coordinates, a geometric constraint equation is constructed using the cosine theorem. Solving this equation yields the 3D local coordinates of the encoded point in the camera coordinate system. Then, singular value decomposition is used to calculate the rotation matrix and translation vector between world and camera coordinates, achieving accurate 3D pose estimation from 2D observation. This provides independent camera pose estimation for each frame, effectively blocking the error propagation chain at its source. This initial value is directly used as the initial value for subsequent global nonlinear reprojection error optimization, effectively preventing the optimization process from getting trapped in local minima and accelerating algorithm convergence.

[0075] In some embodiments, pose estimation step S4 further includes: The three-dimensional spatial coordinates of the coded points in each matching point pair are reprojected to obtain the reprojected pixels. The reprojection error is obtained by calculating the deviation between the coordinates of the reprojected pixel and the corresponding equivalent air pixel. With the goal of minimizing the reprojection error, the initial camera pose is iteratively optimized to obtain the optimized camera pose.

[0076] By reprojecting the three-dimensional spatial coordinates to obtain the reprojected pixels, the deviation between the reprojected pixels and the equivalent air pixel coordinates is calculated as the reprojection error. The reprojection error of all matching point pairs is minimized through iterative optimization, so that the camera pose parameters reach the optimum in the global range. This effectively suppresses the noise influence introduced in the feature extraction and polygon solving process, and greatly improves the accuracy and numerical stability of pose calculation.

[0077] Furthermore, to eliminate the noise introduced during feature extraction and polynomial solving, all equivalent matching points in the set are utilized. Global nonlinear optimization is performed on the group matching point pairs; Minimizing the reprojection error is used as the objective function for the initial camera pose. The objective function is obtained through optimization and solution, and its form is:

[0078] in, This indicates optimization of camera pose; This is the initial camera pose; For the first The reprojection error of a pair of matching points; For the first Equivalent air pixel coordinates of each encoding point; This is the camera intrinsic parameter matrix; These are normalization parameters; For nonlinear least squares problems ,in This represents any variable that needs to be solved by least squares, which is the initial camera pose to be optimized. It can be solved iteratively. From the given initial value Initially, this initial value is the initial value of the initial camera pose, and in the first... Find the increment in the next iteration. ,make Reaching the minimum value, until If the value is sufficiently small, stop iterating to obtain the optimal solution to the objective function; otherwise, update. Proceed to the next iteration; Therefore, the current goal is to find incremental growth. Make To reach the minimum, Perform a first-order Taylor expansion linearization:

[0079] Among them, the Jacobian matrix for about The derivative; Therefore, it can be solved by constructing a linear least squares problem. :

[0080] in, To optimize incremental growth; Adjust the above equation Taking the derivative and setting it to zero, we obtain the incremental equation:

[0081] In the process of solving camera pose optimization, the key lies in solving the corresponding incremental equations; By calculating the derivative of the reprojection error term with respect to the camera pose, the direction of the optimization gradient is determined, so that the optimization algorithm adjusts the camera pose in the direction that reduces the total reprojection error. The specific implementation is as follows: The reprojected pixels can be obtained from the camera projection model. The following relationship must be satisfied:

[0082] in, For the first The three-dimensional local coordinates of each encoded point in the left camera coordinate system The coordinates of the camera's principal point are obtained through camera intrinsic parameter calibration; During the optimization process, the reprojection error is the equivalent air image point after the encoded point is transformed by the underwater image transformation model. and reprojected pixels The pixel coordinate deviation between them is calculated using the following formula:

[0083] Since differentiation is required later, the initial camera pose needs to be determined. The exponential mapping, i.e. ,in yes The corresponding Lie algebra has the following dimension. Apply small perturbations to the pose Using Taylor series to Performing first-order linearization, then Regarding camera pose increment The linear approximation is:

[0084] Solve for the derivative of the error with respect to the disturbance using the chain rule:

[0085] From the relationship between the variables in the formula for calculating reprojection error, we can obtain:

[0086] Points that can be converted to camera coordinates After initial camera pose Transformed into points in the world coordinate system Its relative to the disturbance The derivative is:

[0087] in, for The antisymmetric matrix formed by the coordinates of ; In summary Jacobian matrix :

[0088] Obtain the Jacobian matrix and reprojection error term Then, substituting into the incremental equation, we get:

[0089] In the above incremental equation, the Levenberg-Marquardt damping term makes the incremental equation become:

[0090] in, This is the damping factor, and its initial value is typically taken as 10. -8 Up to 10 -6 Then, as the algorithm is optimized, it is continuously adjusted to balance the two optimization strategies, Gauss-Newton's method and steepest descent method, when choosing the gradient descent strategy; Represents the extraction matrix A diagonal matrix composed of the diagonal elements; The optimized camera pose is obtained by iteratively calculating using the above incremental equation. First, the initial value of the initial camera pose is given. , using the Camera pose during the next iteration The Jacobian matrix and reprojection error are used to obtain the camera pose increment through incremental equations. If the increment Reduce to a sufficiently small threshold Or the number of iterations has reached the set maximum number of iterations. If the iteration stops, the optimized camera pose is obtained. Otherwise, update the camera pose and utilize the new pose. Continue iteratively calculating the pose increment until the iteration stopping condition is met; For example, the pseudocode for camera pose optimization is as follows: Input: Maximum number of iterations Convergence threshold Initial value of damping factor Initial pose ; Output: Optimized pose ; Initialize the number of iterations ; Initialize damping factor ; Initialize the current pose ; Initialize the target function value This represents the initial reprojection error; while and do for to do; Calculate the first Reprojection error of each encoding point ; Calculate the first Jacobian matrix of encoding points ; end for Constructing incremental equations ; Solving for the increment ; if then the current pose Updated to ,Will Assign to , reduce ; else increase ; ; end while ; return ; By reprojecting the 3D spatial coordinates to obtain reprojected pixels, the deviation between these pixels and the equivalent air pixel coordinates is calculated as the reprojection error. Iterative optimization is then performed with the goal of minimizing the sum of squared reprojection errors for all matching point pairs, enabling the camera pose parameters to reach their global optimum. This effectively suppresses the noise introduced during feature extraction and polynomial solving, significantly improving the accuracy and numerical stability of pose calculation. Starting from the initial pose obtained through singular value decomposition, this optimization process effectively avoids getting trapped in local minima and accelerates algorithm convergence, ultimately outputting a high-precision globally optimized camera pose.

[0091] 3D Reconstruction Step S5: Acquire a line structured light image of the underwater target surface to be tested. After refraction compensation of the line structured light image through an underwater image conversion model, extract the laser center line. Perform 3D reconstruction on the laser center line to obtain a local point cloud. Based on optimized camera pose, fuse and reconstruct the local point cloud to output a 3D surface model.

[0092] In some embodiments, the three-dimensional reconstruction step S5 specifically includes: Acquire line structured light images of the surface of the underwater target to be tested; The center line of the laser bright stripe in the line structured light image is extracted to obtain the initial center line coordinate sequence. The initial center line coordinate sequence is then smoothed to obtain the laser center line.

[0093] By acquiring line structured light images and extracting the center line of the laser bright stripe, and then smoothing the initial center line coordinate sequence, high-frequency noise and pixel jumps caused by underwater suspended particles scattering are effectively filtered out. While preserving the original curved shape of the light stripe, the jagged burrs are eliminated, providing smooth and accurate laser center line features for subsequent stereo matching, and ensuring the purity and geometric accuracy of the underlying data for 3D reconstruction.

[0094] Furthermore, while acquiring environmental images, the current image is simultaneously acquired. A binocular structured light image of the surface of the underwater target under test; Due to the strong scattering of suspended particles in the water medium, the initial light stripe center extracted by the traditional gray-scale centroid method or Steger algorithm often contains a large amount of high-frequency noise and spikes, i.e. local pixel jumps. To obtain sub-pixel level smooth light stripe features, the Savitzky-Golay filtering algorithm is introduced to smooth and correct the initial light stripe center coordinates; Figure 5 This is a schematic diagram of a light stripe center filtering process provided in an embodiment of the present invention, illustrating the overall movement process of a sliding window on a data point sequence, such as... Figure 5As shown in the figure, the horizontal axis represents the row coordinate index of the light bar center point, the vertical axis represents the column coordinate value of the light bar center point, and each dot on the curve represents an initial light bar center point. Its horizontal coordinate represents the row position of the point in the image, and its vertical coordinate represents the column position of the point in the image. The direction of window movement is shown by the arrow. like Figure 6 As shown, the red dots represent the center points of the initial light bars within the current sliding window, and the yellow dots represent the points to be smoothed within the current sliding window. That is, the target point that needs to be smoothed; where Let be the row coordinates of that point. Here are the column coordinates of the point; the data points within the window are labeled sequentially as follows: , ... ... , ; The column coordinate sequence of the initial center As the data to be optimized, the size of the sliding window is set to... The number of data points to be optimized is Then for each center point Take the front and back of each Each center point forms a local window, and within the local window... Each data point uses Polynomial fitting of degree one, taking the value of the fitted curve at the center point of the window as the smoothed coordinates; The fitted curve can be represented as:

[0095] in, The row coordinates of the pixel. The fitted value is the column coordinate of the center of the light bar in that row. The coefficients of the polynomial fitting are to be determined. Each data point can be obtained The equations are combined to form a system of equations:

[0096] Let matrix ; matrix ; The least squares solution for the fitted parameters is:

[0097] in, This is the least squares estimate of the fitted coefficient vector; So the initial center point The smoothed column coordinates can be represented as:

[0098] in, The smoothed column coordinates Let be the row coordinates of that point. These are the estimated values ​​of the polynomial coefficients determined by least squares fitting; Through the above filtering process, the system effectively filters out high-frequency noise while perfectly preserving the original peak and edge morphology of the light stripe, ultimately obtaining a high-precision laser centerline. The filtered and smoothed laser centerline provides high-quality underlying data for subsequent 3D reconstruction.

[0099] In some embodiments, the three-dimensional reconstruction step S5 further includes: Perform stereo matching on the laser center lines along the epipolar direction to determine the matching pixel pairs between each laser center line; Based on the positional differences between pixels in a pixel pair, the three-dimensional spatial coordinates corresponding to each pixel are calculated to obtain a local point cloud.

[0100] By performing stereo matching along the epipolar direction of the laser centerline, corresponding pixel pairs in the optical structure image are determined. Based on the positional differences of the pixel pairs, the three-dimensional spatial coordinates are calculated through triangulation, thus realizing the accurate reconstruction from two-dimensional laser stripes to three-dimensional spatial points. This overcomes the problem of sparse single-frame point clouds in underwater scattering environments and provides high-precision local three-dimensional data for subsequent multi-frame point cloud fusion.

[0101] Furthermore, extract the first A binocular linear structured light image of the target surface, comprising a left linear structured light image and a right linear structured light image, which correspond to the target surface images captured by the left and right cameras of the underwater binocular system at the same time. Using an underwater image conversion model, refraction compensation is performed on the two-dimensional pixel coordinates of each pixel in the left and right line structured light images to obtain the left and right equivalent line structured light images. Through this compensation, the epipolar bending distortion caused by underwater refraction is eliminated, and the epipolar lines in the left and right equivalent line structured light images are restored to standard straight lines, thereby satisfying the straight epipolar line constraint of binocular stereo vision in air. The left laser center line and the right laser center line are extracted from the compensated left and right equivalent line structured light images, respectively. Perform stereo matching on the left and right laser center lines along the epipolar direction to determine the matching pixel pairs between the left and right laser center lines, i.e., the same-name point pairs. Specifically, for each pixel on the left laser center line, the pixel with the closest gray value is searched along the corresponding epipolar direction on the right laser center line as a matching point, and the two form a pair of corresponding points. Based on the positional difference between the left and right pixels in a pair of corresponding points, the three-dimensional spatial coordinates of each pixel are calculated using the principle of triangulation. The basic principle of binocular triangulation is that, due to the known baseline distance and relative orientation between the left and right cameras, the imaging position of the same object point in space is laterally offset in the left and right images. This offset is inversely proportional to the depth of the object point from the camera. Based on the intrinsic parameter matrices of the left and right cameras, the extrinsic parameters of the binocular system, and the disparity values ​​of corresponding point pairs, the three-dimensional spatial coordinates of the object point in the coordinate system of the left camera can be reconstructed using the principle of triangulation. Performing the above triangulation calculations on each pair of corresponding points yields a local point cloud composed of multiple three-dimensional spatial points. This local point cloud describes the three-dimensional topography of the target surface in the current frame; Through the above process, accurate reconstruction from two-dimensional laser stripes to three-dimensional spatial points was achieved, overcoming the problem of sparse single-frame point clouds in underwater scattering environments, and providing high-precision local three-dimensional data for subsequent multi-frame point cloud fusion.

[0102] In some embodiments, the three-dimensional reconstruction step S5 further includes: Based on the optimized camera pose, coordinate transformation is performed on the local point cloud to obtain the global point cloud; The global point cloud is accumulated to obtain the complete point cloud. After filtering and denoising the complete point cloud, a continuous triangular mesh is generated through a surface reconstruction algorithm to obtain a three-dimensional surface model.

[0103] By optimizing the camera pose, the local point clouds of each frame are transformed to the same coordinate system for accumulation. The complete point cloud is then filtered, denoised, and reconstructed. This allows each frame of point cloud to be stitched together directly based on a global absolute reference, fundamentally eliminating the inherent defect of error accumulation with distance in traditional inter-frame recursive methods. Ultimately, a high-precision continuous 3D surface model without geometric distortion and faults is generated.

[0104] Furthermore, as the mobile platform moves with the binocular system, the local point cloud in the left camera coordinate system of the current frame is rigidly transformed based on the optimized camera poses of each frame. Specifically, for local point clouds For each 3D point in the image, the camera pose is optimized. ,in, This is the optimized rotation matrix; This is the optimized translation vector; Transform the point from the left camera coordinate system to the global scale coordinate system to obtain the corresponding coordinates of the point in the global scale coordinate system; By performing the above transformation operation on all 3D points in the local point cloud, the local point cloud of the current frame can be transformed to the global scale coordinate system, thus obtaining the global point cloud of the current frame. ;in, ; Since the optimized camera pose for each frame is solved independently relative to the stationary global scale and does not depend on the calculation results of the previous frame, no error from the previous frame is introduced during the coordinate transformation of the point cloud in each frame, which fundamentally blocks the accumulation chain of inter-frame errors. As mobile platforms equipped with underwater binocular systems continue to advance, the system repeatedly performs the above-mentioned operations of code point extraction, refraction compensation, pose estimation, local point cloud reconstruction, and global coordinate transformation for each frame. After each frame is processed, the global point cloud obtained after the transformation of the current frame is added to the existing complete point cloud, gradually expanding the coverage area. During the accumulation process, since the point clouds of each frame are located in a unified global scale coordinate system, no additional registration operation is required between different frames to directly superimpose them, thus avoiding the model distortion problem caused by accumulated errors in traditional point cloud stitching methods. After scanning, the system obtains a large-scale original complete point cloud covering the entire surface of the underwater target to be tested. Due to factors such as underwater suspended particles scattering and uneven illumination, this complete point cloud may contain a small number of scattered outlier noise points. The system uses a statistical filtering algorithm to process the original complete point cloud. This algorithm statistically analyzes the spatial distribution characteristics of each point's neighborhood, identifies points that deviate far from the main point cloud distribution as noise points and removes them, thus obtaining a denoised point cloud. After denoising, the system performs surface reconstruction on the denoised point cloud based on the Poisson surface reconstruction algorithm. The Poisson surface reconstruction algorithm transforms the point cloud surface reconstruction problem into solving the Poisson equation. Through steps such as constructing an octree structure, defining a vector field, solving the scalar indicator function, and extracting isosurfaces, the discrete three-dimensional point cloud is encapsulated into a continuous triangular mesh surface. Finally, the system outputs a high-precision, continuous three-dimensional surface mesh model of the underwater target under test.

[0105] Through the above process, based on the optimized camera pose, the local point clouds of each frame are rigidly transformed to the same coordinate system for accumulation, and the complete point cloud is filtered, denoised and reconstructed. This allows each frame of point cloud to be directly stitched together based on the global absolute reference, fundamentally eliminating the inherent defect of error accumulation with distance in the traditional inter-frame recursive method, and finally generating a high-precision continuous three-dimensional surface model without geometric distortion and discontinuity.

[0106] Based on the above-mentioned underwater scene 3D reconstruction method, by pre-measuring the 3D spatial coordinates of all coding points on the coding scale, extracting the 2D pixel coordinates of the coding points in the image and pairing them with the 3D spatial coordinates, a known absolute reference is provided for subsequent positioning. By establishing an underwater image conversion model to compensate for refraction distortion, distorted pixels are converted into equivalent air pixels, avoiding the divergence problem of the traditional PnP algorithm due to refraction nonlinearity. This ensures that the two-dimensional and three-dimensional matching relationship strictly meets the perspective projection constraints, providing an accurate data foundation for pose solving. Furthermore, local point clouds were obtained through 3D reconstruction of line structured light images, and fusion reconstruction was performed based on optimized camera pose, achieving high-precision underwater 3D reconstruction without relying on inertial navigation or acoustic positioning equipment.

[0107] It should be noted that the above is a reference method for underwater scene 3D reconstruction based on a coding scale, and the present invention is not limited thereto.

[0108] The embodiments of the present invention realize low-cost and highly robust underwater mobile 3D reconstruction, and solve the technical problems in the prior art that rely on expensive inertial navigation or acoustic positioning equipment, visual positioning is prone to failure underwater, and inter-frame recursion inevitably leads to cumulative drift error, resulting in geometric distortion and discontinuity of the 3D model during long-distance scanning, which cannot meet the technical requirements of high-precision industrial measurement.

[0109] Finally, it should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0110] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A method for 3D reconstruction of underwater scenes based on a coding scale, characterized in that, include: Ruler setup steps: Fix the coded ruler with coded points printed on it to the underwater target to be measured, and measure the three-dimensional spatial coordinates of all coded points on the coded ruler; Point set establishment steps: Acquire environmental images containing the coded scale and the underwater target to be measured through a camera, extract the two-dimensional pixel coordinates of the coded points in the environmental images, and pair the two-dimensional pixel coordinates and the three-dimensional spatial coordinates of the same coded point to obtain a matching point set; Refraction compensation step: Obtain the optical parameters of the camera, establish an underwater image conversion model based on the optical parameters, and convert the two-dimensional pixel coordinates in the matching point set into equivalent air pixel coordinates through the underwater image conversion model to obtain an equivalent matching point set; Pose estimation steps: Select matching point pairs from the equivalent matching point set, solve the geometric constraints for each selected matching point pair to obtain the three-dimensional local coordinates of the encoded points in each matching point pair, calculate the initial camera pose based on the three-dimensional spatial coordinates and the three-dimensional local coordinates, and optimize the initial camera pose to obtain the optimized camera pose; 3D Reconstruction Steps: Acquire a line structured light image of the surface of the underwater target to be tested. After refraction compensation of the line structured light image using the underwater image conversion model, extract the laser center line. Perform 3D reconstruction on the laser center line to obtain a local point cloud. Based on the optimized camera pose, fuse and reconstruct the local point cloud to output a 3D surface model.

2. The underwater scene 3D reconstruction method based on a coding scale according to claim 1, characterized in that, The specific steps for setting up the scale are as follows: A world coordinate system is established with the center of the encoding point at a preset position on the encoding scale as the origin, the length direction of the encoding scale as the X-axis, the width direction of the encoding scale as the Y-axis, and the direction perpendicular to the plane of the encoding scale as the Z-axis. The three-dimensional spatial coordinates of all coding points in the world coordinate system are measured using a coordinate measuring machine, and the code value and its three-dimensional spatial coordinates corresponding to each coding point are stored in a lookup table.

3. The underwater scene 3D reconstruction method based on a coding scale according to claim 2, characterized in that, The specific steps for establishing the point set are as follows: Extract the center position of the encoded points in the environmental image to obtain the two-dimensional pixel coordinates of each encoded point on the environmental image; The extracted encoding points are decoded to obtain the code value corresponding to the encoding point. Based on the code value, the corresponding three-dimensional spatial coordinates are retrieved from the lookup table. The matching point set is established based on the three-dimensional spatial coordinates and the two-dimensional pixel coordinates of the same encoding point.

4. The underwater scene 3D reconstruction method based on a coding scale according to claim 1, characterized in that, The refractive compensation step is specifically as follows: The equivalent focal length of the camera, the unit normal vector of the waterproof glass plane outside the camera, and the distance from the optical center of the camera to the waterproof glass plane are obtained as the optical parameters. Based on the optical parameters, calculate the coordinates of the intersection point between the light beam projected from the pixel corresponding to the two-dimensional pixel coordinates and the plane of the waterproof cover glass. Based on the intersection coordinates, the mapping relationship between the two-dimensional pixel coordinates and the equivalent air pixel coordinates is determined by the law of refraction, thus obtaining the underwater image conversion model.

5. The underwater scene 3D reconstruction method based on a coding scale according to claim 4, characterized in that, The refractive compensation step further includes: Based on the aforementioned optical parameters, the refractive index scaling factor is derived using the law of refraction. Obtain the approximate depth of the encoded points in the matching point set. Based on the refractive index scale factor and the approximate depth, convert the two-dimensional pixel coordinates of each encoded point in the matching point set into the corresponding equivalent air pixel coordinates to obtain the equivalent matching point set.

6. The underwater scene 3D reconstruction method based on a coding scale according to claim 1, characterized in that, The pose estimation step is specifically as follows: Based on the equivalent air pixel coordinates, calculate the cosine value of the angle between the optical center of the camera and the line of sight to the coding point of each matching point pair; Based on the three-dimensional spatial coordinates and the cosine value of the included angle, a geometric constraint equation is constructed using the cosine theorem. The geometric constraint equation is then solved to obtain the three-dimensional local coordinates of the coding points in each matching point pair. Based on the three-dimensional spatial coordinates and three-dimensional local coordinates of the coded points in each matching point pair, the rotation matrix and translation vector are calculated through singular value decomposition to obtain the initial camera pose.

7. The underwater scene 3D reconstruction method based on a coding scale according to claim 1, characterized in that, The pose estimation step further includes: The three-dimensional spatial coordinates of the encoded points in each of the matching point pairs are reprojected to obtain the reprojected pixels. The deviation between the reprojected pixel and the corresponding equivalent air pixel coordinates is calculated to obtain the reprojection error; With the goal of minimizing the reprojection error, the initial camera pose is iteratively optimized to obtain the optimized camera pose.

8. The underwater scene 3D reconstruction method based on a coding scale according to claim 1, characterized in that, The specific steps of the three-dimensional reconstruction are as follows: Acquire a line structured light image of the surface of the underwater target to be tested; The center line of the laser bright stripe in the line structured light image is extracted to obtain an initial center line coordinate sequence. The initial center line coordinate sequence is then smoothed to obtain the laser center line.

9. The underwater scene 3D reconstruction method based on a coding scale according to claim 8, characterized in that, The three-dimensional reconstruction step also includes: The laser center lines are stereo matched along the epipolar direction to determine the matching pixel pairs between each laser center line; Based on the positional differences between pixels in the pixel pair, the three-dimensional spatial coordinates corresponding to each pixel are calculated to obtain the local point cloud.

10. The underwater scene 3D reconstruction method based on a coding scale according to claim 1, characterized in that, The three-dimensional reconstruction step also includes: Based on the optimized camera pose, the local point cloud is transformed to obtain the global point cloud; The global point cloud is accumulated to obtain a complete point cloud. After filtering and denoising the complete point cloud, a continuous triangular mesh is generated by a surface reconstruction algorithm to obtain the three-dimensional surface model.

Images