Ship lock uncalibrated 3D reconstruction method based on double view and DUSt3R

Abstract

The application provides a ship lock non-calibration three-dimensional reconstruction method based on double-view and DUSt3R, comprising: S1. double-view image acquisition; S2. double-view image preprocessing; S3. DUSt3R algorithm and double-view feature processing; S4. three-layer coordinate mapping model construction and point cloud coordinate conversion; S5. three-dimensional grid model generation and geographic positioning fusion optimization; S6. error correction and performance verification; S7. digital twin platform integration and application. The application aims to solve the problems of complex three-dimensional reconstruction process, high cost, poor real-time performance, dependence on manual operation and scheduling, and insufficient safety supervision of traditional ship lock.

Description

Technical Field

[0001] This invention relates to the field of intelligent water transport and intelligent transportation technologies, and in particular to a calibration-free 3D reconstruction method for ship locks based on dual views and DUSt3R. Background Technology

[0002] Water transport, as an important component of the comprehensive transportation system, occupies a key position in the transportation of bulk commodities. As a core hub of inland waterways, the operational efficiency and safety level of locks directly affect the waterway's traffic capacity.

[0003] Traditional 3D reconstruction technology for ship locks has significant limitations: it relies on complex calibration processes using multiple sensors, requiring specialized equipment and personnel, resulting in high implementation costs; the reconstruction process is time-consuming, failing to meet the business needs of real-time ship lock scheduling; furthermore, traditional ship lock operation relies on manual experience-based scheduling, resulting in insufficient vessel positioning accuracy, and safety supervision is mainly based on two-dimensional monitoring, making it difficult to intuitively identify risks such as vessels exceeding limits or crossing boundaries, and failing to form a closed-loop management system throughout the entire process. With the policy guidance of digital transformation of transportation infrastructure, traditional technologies are no longer suitable for the "efficient, safe, and intelligent" construction requirements of smart ship locks. There is an urgent need for a low-cost, high-efficiency, calibration-independent 3D ship reconstruction and geolocation technology to provide core support for the digital operation of ship locks. Summary of the Invention

[0004] This invention provides a calibration-free 3D reconstruction method for ship locks based on dual-view and DUSt3R, aiming to solve the pain points of traditional ship lock 3D reconstruction processes, such as complexity, high cost, poor real-time performance, reliance on manual operation scheduling, and insufficient safety supervision. By integrating technologies such as artificial intelligence, computer vision, and geographic information systems, it achieves high-precision, fast, and low-cost 3D reconstruction and accurate geolocation of ships, and constructs a closed-loop system of "perception-modeling-application-feedback" to support the full-element digital expression, safe and efficient operation management, refined maintenance, and scientific decision-making of smart ship locks. This promotes the intelligent upgrading of inland waterway ship locks and provides technical support for the construction of a strong transportation nation and smart water transport.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A calibration-free 3D reconstruction method for locks based on dual views and DUSt3R includes:

[0007] S1. Simultaneously acquire front and side view images of navigating vessels to form a dual-view image pair;

[0008] S2. The dual-view image pair is sequentially subjected to noise suppression, reflection suppression and distortion correction to obtain a preprocessed dual-view image pair;

[0009] S3. Input the preprocessed dual-view image into the trained DUSt3R algorithm model based on the Transformer architecture, extract the deep features of the dual-view image, perform feature matching using a feature matching mechanism, and estimate the dense point cloud based on the effective matching feature pairs obtained by feature matching through a triangulation algorithm.

[0010] S4. Set up multiple ground control points in the gate chamber area, obtain the pixel coordinates, gate chamber coordinates and geodetic latitude and longitude coordinates of each control point, construct an image pixel-gate chamber coordinate system mapping model and a gate chamber coordinate system-geodetic latitude and longitude coordinate system mapping model, and convert the dense point cloud into a dense point cloud in the geodetic latitude and longitude coordinate system in sequence through the two mapping models.

[0011] S5. Denoise and downsample the dense point cloud in the geodetic latitude and longitude coordinate system. Based on the processed point cloud, generate a three-dimensional mesh model through a surface reconstruction algorithm. Associate the three-dimensional mesh model with the geographic coordinate system to achieve geographic positioning fusion and obtain the geographically positioned three-dimensional mesh model.

[0012] S6. Obtain the actual dimensions and actual positioning coordinates of the ship, and calculate the size measurement error and positioning error of the three-dimensional mesh model after geolocation. If the error exceeds the corresponding preset error threshold, adjust the relevant algorithm parameters and repeat steps S3-S5 until the error meets the requirements, and obtain the three-dimensional mesh model after error correction.

[0013] This specification also includes the following method for calibration-free 3D reconstruction of locks based on dual views and DUSt3R:

[0014] S7. Convert the error-corrected 3D mesh model into a format supported by the digital twin platform, import it through the platform interface and link it with the real-time data of the lock to realize the visualization and business application of the model.

[0015] In this specification, in S3, the feature extraction module of the DUSt3R algorithm model adopts a multi-layer Transformer encoder. Each encoder layer includes a multi-head self-attention layer and a feedforward neural network layer. The multi-head self-attention layer captures the long-distance dependencies between image pixels in parallel through multiple sets of attention heads. The feedforward neural network layer performs dimensional transformation and enhancement on the extracted features through a non-linear activation function.

[0016] In this specification, in S3, the feature matching mechanism uses cross-correlation attention to calculate the matching similarity of deep features in the two views, and filters valid matching feature pairs through a dynamic matching degree threshold. The dynamic matching degree threshold is adaptively adjusted according to the illumination changes and ship attitude differences in the two view images, and false matching features are eliminated through neighborhood feature consistency verification.

[0017] In this specification, in S4, both the image pixel-gate chamber coordinate system mapping model and the gate chamber coordinate system-geodetic latitude and longitude coordinate system mapping model adopt affine mapping models. The mapping matrix parameters are solved by the least squares method. During the solution process, a weight allocation strategy for control point coordinates is introduced to give higher weights to the control points in the key areas of the gate chamber, thereby realizing the conversion from gate chamber coordinates to geodesic latitude and longitude coordinates, establishing the association between gate chamber coordinates and geodesic latitude and longitude coordinates, and enabling the point cloud to have geographic positioning capabilities.

[0018] In this specification, in step S5, the denoising uses a statistical filtering algorithm. By calculating the average distance and standard deviation of a preset number of neighboring points for each point, points that exceed the range of the average distance and the preset multiple of the standard deviation are identified as outliers and removed. The downsampling process uses a voxel grid downsampling algorithm. The voxel size is dynamically adapted according to the density characteristics of the dense point cloud and the accuracy requirements of the three-dimensional mesh model.

[0019] In this specification, in S3, the triangulation algorithm is based on the epipolar geometric constraints of dual views. It constructs a system of linear equations to solve the three-dimensional spatial coordinates by projecting the corresponding feature points in the coordinate systems of the two cameras. During the solution process, a robust estimation strategy is introduced to suppress the influence of matching errors on the solution results of the three-dimensional point cloud coordinates.

[0020] In this specification, S6, the relevant algorithm parameters include the number of feature extraction network layers of the DUSt3R algorithm model, the feature matching threshold, the robust estimation coefficient of the triangulation algorithm, the control point weights of the coordinate mapping model, and the resolution parameters of the surface reconstruction algorithm. The parameter adjustment is optimized in a directional manner according to the distribution characteristics of the size measurement error and the positioning error until the error meets the preset requirements.

[0021] In this specification, in S5, the surface reconstruction algorithm adopts the Poisson surface reconstruction algorithm. During the reconstruction process, the normal vector of the point cloud is estimated by fitting a plane through neighborhood points and the direction of the normal vector is unified. Then, an implicit function is constructed based on the point cloud and the normal vector. A triangular mesh model is generated by extracting the isosurface of the preset implicit function value. The smoothness parameter of the implicit function is adaptively set according to the density of the point cloud.

[0022] In this specification, in S2, the noise suppression uses a Gaussian filtering algorithm, and the size of the Gaussian kernel function is dynamically selected according to the image noise intensity; the reflection suppression uses the Otsu method to solve the adaptive threshold segmentation of the reflection area, performs brightness compression processing on the reflection area, and performs histogram equalization on the non-reflective area to enhance feature contrast; the distortion correction uses a quadratic polynomial distortion model, obtains the distortion coefficients through a standard calibration template, and accurately corrects the radial distortion of the image.

[0023] In summary, the present invention has at least the following beneficial effects:

[0024] By eliminating reliance on specialized calibration equipment and personnel, simplifying the 3D reconstruction process, reducing the implementation cost and technical threshold of intelligent transformation of ship locks, and promoting the large-scale popularization of 3D visualization technology in industrial scenarios.

[0025] This enables rapid 3D reconstruction and precise geolocation of ships, providing intuitive spatial data support for intelligent lock scheduling, optimizing lock chamber resource allocation, improving operational efficiency, and promoting the transformation of lock operation from "human experience-driven" to "data intelligence-driven".

[0026] Building a comprehensive safety supervision system can intuitively identify risks such as ships exceeding limits or crossing boundaries, strengthen the tracking of key vessels throughout the entire process, improve the safety guarantee capacity of lock navigation, and reduce potential safety hazards.

[0027] By deeply integrating with the digital twin platform, real-time visualization and linkage of ship status, lock chamber environment, and equipment operation can be achieved, improving the transparency and precision of lock operation and management, and providing reliable data support for decision analysis.

[0028] Reducing ship waiting time and idling reduces carbon emissions, aligns with the requirements of green transportation development, and has both economic and social benefits. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a schematic diagram illustrating the steps of the calibration-free 3D reconstruction method for ship locks based on dual views and DUSt3R involved in this invention.

[0031] Figure 2 This is a schematic diagram of the process of the calibration-free 3D reconstruction method for locks based on dual views and DUSt3R involved in this invention.

[0032] Figure 3 This is a schematic diagram of the DUSt3R processing flow involved in this invention.

[0033] Figure 4 This is a schematic diagram of the coordinate mapping and 3D mesh generation process involved in this invention. Detailed Implementation

[0034] In the following text, only some exemplary embodiments are briefly described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the embodiments of the present invention. Therefore, the accompanying drawings and the description are considered to be exemplary in nature rather than restrictive.

[0035] The following disclosure provides many different embodiments or examples for implementing different structures of the embodiments of the present invention. To simplify the disclosure of the embodiments of the present invention, the components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the embodiments of the present invention. In addition, the embodiments of the present invention may repeat reference numerals and / or reference letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0036] This solution takes digital twin technology as the core carrier and proposes a calibration-free 3D reconstruction technology for ship locks based on dual views and DUSt3R. By deploying industrial cameras on both sides of the lock chamber, collecting the front and side view images of the ship and optimizing them through preprocessing, and then inputting them into the DUSt3R algorithm model based on the Transformer architecture, the dense point cloud and camera pose can be directly estimated without calibrating the internal and external parameters of the camera; a three-layer coordinate mapping model of "image pixel - lock chamber coordinate system - geodetic longitude and latitude" is constructed, and through the calibration of a small number of ground control points, the geographic coordinate transformation of the point cloud is realized; a high-precision 3D mesh model is generated based on the transformed geodetic coordinate point cloud, and after error correction and performance verification, it is integrated into the digital twin platform to realize the linkage display of the model and the real-time data of the ship lock, providing full-process digital support for the intelligent scheduling, safety supervision, and operation analysis of the ship lock.

[0037] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0038] As Figure 1 and Figure 2 shown, this embodiment provides a calibration-free 3D reconstruction method for ship locks based on dual views and DUSt3R, including:

[0039] S1. Synchronously collect the front view image and side view image of the navigating ship to form a dual view image pair;

[0040] S2.依次对所述双视图图像对进行噪声抑制、反光抑制和畸变校正,得到预处理后的双视图图像对;

[0041] S3. Input the preprocessed dual-view image into the trained DUSt3R algorithm model based on the Transformer architecture, extract the deep features of the dual-view image, perform feature matching using a feature matching mechanism, and estimate the dense point cloud based on the effective matching feature pairs obtained by feature matching through a triangulation algorithm.

[0042] S4. Set up multiple ground control points in the gate chamber area, obtain the pixel coordinates, gate chamber coordinates and geodetic latitude and longitude coordinates of each control point, construct an image pixel-gate chamber coordinate system mapping model and a gate chamber coordinate system-geodetic latitude and longitude coordinate system mapping model, and convert the dense point cloud into a dense point cloud in the geodetic latitude and longitude coordinate system in sequence through the two mapping models.

[0043] S5. Denoise and downsample the dense point cloud in the geodetic latitude and longitude coordinate system. Based on the processed point cloud, generate a three-dimensional mesh model through a surface reconstruction algorithm. Associate the three-dimensional mesh model with the geographic coordinate system to achieve geographic positioning fusion and obtain the geographically positioned three-dimensional mesh model.

[0044] S6. Obtain the actual dimensions and actual positioning coordinates of the ship, and calculate the size measurement error and positioning error of the three-dimensional mesh model after geolocation. If the error exceeds the corresponding preset error threshold, adjust the relevant algorithm parameters and repeat steps S3-S5 until the error meets the requirements, and obtain the three-dimensional mesh model after error correction.

[0045] In some embodiments, the calibration-free 3D reconstruction method for locks based on dual views and DUSt3R further includes:

[0046] S7. Convert the error-corrected 3D mesh model into a format supported by the digital twin platform, import it through the platform interface and link it with the real-time data of the lock to realize the visualization and business application of the model.

[0047] The core technology implementation plan and principles of each step will be described in detail below.

[0048] S1. Dual-view image acquisition

[0049] This step serves as the foundation for the data source of 3D reconstruction. Its core objective is to acquire clear and synchronized frontal and side views of the ship by deploying dual industrial cameras in a standardized manner. This provides high-quality raw data for subsequent algorithm processing and solves the problems of asynchronous data acquisition and incomplete scene coverage in traditional multi-sensor acquisition.

[0050] 1. Camera Selection and Deployment Specifications: An industrial camera with a resolution of 4096×2160 pixels (model: Basler acA4112-30uc) was selected. This camera features a global shutter function to avoid motion blur and meet the dynamic shooting requirements during ship navigation. A front-view camera was deployed perpendicular to the navigation centerline on the left side of the lock chamber, and a side-view camera was deployed parallel to the navigation centerline on the right side. Both cameras were installed at a uniform height of 10m above the water surface, with a lens focal length of 25mm, ensuring that the acquisition range completely covers the effective navigation area of ​​the lock chamber (200m long × 30m wide) without any blind spots. Stainless steel brackets were used for installation, with the bottom of the brackets connected to the concrete structure of the lock chamber using expansion bolts to ensure a wind resistance level of ≥10, preventing camera displacement due to water flow vibration or wind force.

[0051] 2. Collaborative Configuration of Acquisition Parameters: To ensure consistency between the two view images, both cameras are configured with the following parameters: frame rate 30fps, exposure time 1 / 500s (to avoid overexposure due to water surface reflections), ISO sensitivity 400 (to balance noise and image quality), white balance set to "cloudy mode" (to adapt to the complex lighting conditions in the gate area), and images saved in 16-bit RAW format to retain more image details. All parameters are remotely synchronized and configured using industrial camera management software (Basler Pylon Viewer). After configuration, the parameters are locked to prevent accidental data acquisition errors.

[0052] 3. Data Acquisition Triggering and Synchronization Control: A laser sensor (model: LS-200, detection distance 0-50m, response time ≤1ms) is used as the trigger source and installed 50m upstream of the lock chamber entrance. When the bow of the vessel crosses the laser detection line, the sensor outputs a high-level trigger signal, synchronously controlling two cameras to start data acquisition via the IO interface. The acquisition process lasts for 2 seconds, acquiring a total of 60 frames (30 frames per camera) to ensure the stable attitude of the vessel in the lock chamber is captured.

[0053] 4. Image Filtering and Output: The acquired images are filtered based on their sharpness. Total Variation (TV) is used for quantitative evaluation, and the TV value is calculated using the following formula: Where W and H are the image width and height, and I(u,v) is the grayscale value of pixel (u,v). Images with a resolution of TV≥30 are selected as valid images, and one frame with the highest resolution from each camera is retained to form a dual-view image pair. The front view image is defined as... Its pixel coordinate system is ( The origin is the top left corner of the image. The horizontal pixel axis has a range of [0, 4095]. The vertical pixel axis ranges from [0, 2159]); the side view image is... The pixel coordinate system is (Definition Reference) The final output is a pair of dual-view images in PNG format. The file is named "Ship ID_Acquisition Time_View Type.png" and stored in the image cache directory of the local server to provide input data for the S2 preprocessing step.

[0054] S2. Dual-view image preprocessing

[0055] This step addresses interference factors in the original image, such as noise, water reflection, and lens distortion, aiming to improve image quality and provide clean input features for the DUSt3R algorithm, preventing interference factors from degrading the accuracy of 3D reconstruction. The preprocessed image must maintain the integrity of the ship's outline, clear details, and ensure the consistency of the geometric relationships between the two views.

[0056] 1. Gaussian Filter Noise Suppression: The original image is susceptible to circuit noise and ambient light fluctuations during acquisition, resulting in salt-and-pepper noise and Gaussian noise. A 3×3 Gaussian filter is used for noise reduction. The filtering formula is:

[0057] ;

[0058] in , respectively corresponding , ; Let be the Gaussian kernel function, with the expression: , (These parameters were determined through multiple experiments, effectively filtering out noise while preserving ship edge details.) For image boundary pixels, a mirror-fill method is used to avoid boundary effects, ensuring that the filtered image edges are distortion-free. The core function of this algorithm is to smooth noise interference by weighted averaging of neighboring pixel values, while maintaining the integrity of the ship's main structure, laying the foundation for subsequent feature extraction.

[0059] 2. Adaptive Reflection Suppression: The water surface in the lock area is prone to reflection, leading to overexposure in localized areas of the vessel and loss of detail. A combined method of "threshold segmentation + brightness equalization" is used to address this.

[0060] First, calculate the image grayscale histogram. ( (For grayscale values), adaptive thresholding is solved using Otsu's method. The core idea of ​​Otsu's method is to find a threshold that divides the image into foreground (ships) and background (water), maximizing the inter-class variance between the two parts. The formula is as follows: ,in The percentage of pixels with grayscale ≤ g. This represents the average gray level of the corresponding region. , is the average gray level of the region where gray level is greater than g.

[0061] For grayscale values ​​greater than Brightness compression is applied to reflective areas: To avoid loss of detail in reflective areas, histogram equalization is applied to non-reflective areas to stretch the grayscale range and improve image contrast. The formula is as follows: The advantage of this method lies in its ability to adapt to different lighting conditions and reflective intensity, specifically addressing overexposed areas while ensuring uniform brightness of the ship's main body.

[0062] 3. Polynomial Distortion Correction: Industrial camera lenses exhibit radial distortion, which can deform the ship's silhouette and affect the accuracy of subsequent feature matching. A quadratic polynomial distortion correction model is used to correct this. The correction formula is:

[0063] ;

[0064] ;

[0065] in These are the pixel coordinates of the distorted image. The coordinates are after correction; As the principal point of the image, take the coordinates of the image center. ; The distance from the pixel to the principal point; , The distortion coefficients (obtained through standard checkerboard calibration) , , , The core contribution of distortion correction is to restore the true geometry of the ship, ensuring the accurate positional relationship of corresponding feature points in the dual views, and providing a guarantee for the dimensional accuracy of subsequent 3D reconstruction.

[0066] 4. Data Output: After noise suppression, reflection suppression, and distortion correction, the preprocessed dual-view image pair is obtained. The image size remains 4096×2160 pixels, the storage format is still PNG, and it is directly passed into the S3 DUSt3R algorithm model for feature processing.

[0067] S3.DUSt3R Algorithm and Dual-View Feature Processing

[0068] This step is the core of 3D reconstruction. By constructing and training a DUSt3R model based on the Transformer architecture, feature extraction and matching are performed on the preprocessed dual-view images to estimate dense point clouds and camera pose. This eliminates the dependence on camera intrinsic and extrinsic parameter calibration in traditional 3D reconstruction, achieving calibration-free and rapid reconstruction. The DUSt3R processing flow is as follows: Figure 3 As shown.

[0069] 1. Model Architecture Design:

[0070] Input layer: Receives preprocessed dual-view image pairs output from S2. First, the image size is resized to 2048×1080 pixels (balancing processing speed and accuracy), and then normalized to the [0,1] interval to eliminate the impact of pixel value range differences on model training.

[0071] Feature extraction module: A 6-layer Transformer encoder network is used for feature extraction. Each layer contains a multi-head self-attention layer (8 heads, attention dimension 256) and a feedforward neural network layer (hidden layer dimension 1024). The core function of the multi-head self-attention layer is to capture long-distance dependencies between image pixels, effectively extracting global features (such as hull outline and mast structure) and local features (such as hull details and cargo loading marks) of the ship. The feedforward neural network layer enhances feature representation through nonlinear transformation. The feature extraction formula is as follows: ( ),in This is the deep feature map (dimensions 256×1080×2048) of the k-th image. These are the encoder weight parameters.

[0072] Feature matching module: Employs a cross-correlation attention mechanism to calculate the matching degree of features between two views. This mechanism effectively addresses feature offset issues caused by changes in ship attitude and perspective differences. The matching formula is as follows:

[0073] ;

[0074] in For feature channel index, , This represents the pixel position in the dual-view diagram. A matching threshold of 0.7 is set to filter out valid matching feature pairs. ( , To match the logarithm of features, the average ), eliminating false matches caused by interference from water surface reflections, floating objects, etc.

[0075] Point cloud and pose estimation module: Based on effective matching feature pairs, it calculates 3D point clouds through triangulation algorithms and simultaneously estimates camera pose (rotation matrix). Translation vector The core principle of the triangulation algorithm is to utilize the geometric constraints of the two views and solve for the 3D spatial coordinates by the projection relationship of corresponding feature points in the two camera coordinate systems. The formula is:

[0076] ;

[0077] in For the m-th 3D point (in pixel coordinates), the corresponding projection constraint formula is:

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] In the formula ( ) is a rotation matrix that describes the pose of the camera coordinate system; This is a translation vector, describing the position of the camera in the coordinate system; To match the pixel coordinates of the feature point in the k-th image.

[0083] Output layer: Output dense point cloud and camera pose Point cloud density ≥ 50 points / This ensures complete coverage of the ship's surface details.

[0084] 2. Model training process:

[0085] Dataset Construction: Two-view image pairs were collected from 500 different types of ships (cargo ships, container ships, dangerous goods ships, passenger ships, and engineering ships). Each ship corresponds to 10 sets of images with different navigation attitudes. Simultaneously, high-precision 3D point clouds of the ships were acquired using a laser scanner (model: Faro Focus S70) as ground truth labels, constructing a dataset containing 5000 samples. The dataset was divided into a 7:2:1 ratio: training set (3500 sets), validation set (1000 sets), and test set (500 sets). The training set was used for model parameter learning, the validation set for hyperparameter tuning, and the test set for evaluating the final model performance.

[0086] Loss function design: The total loss function is a weighted sum of the reprojection error loss and the point cloud density loss, as shown in the formula: ,in , These are the weighting coefficients (determined through cross-validation, prioritizing reprojection accuracy). Reprojection error loss. The formula is used to measure the deviation between the estimated point cloud projected back into the image and the original feature points. , , To estimate the reprojection coordinates of the point cloud; point cloud density loss. To ensure uniform point cloud coverage, the formula is: , For the estimated point cloud that is ≤0.1m away from the true point cloud, For realistic point clouds, Indicates the number of elements in the set.

[0087] Training parameter configuration: The AdamW optimizer was used, with an initial learning rate of 0.0001 and a weight decay coefficient of 0.0001. The learning rate decayed to 0.9 every 10 epochs. The batch size was 4, and the training duration was 50 epochs. The training equipment consisted of a GPU cluster (two NVIDIA A100 GPUs, each with 80GB of VRAM), and mixed-precision training was used to improve training speed. The training objective was to validate a weight projection error ≤ 1.5 pixels and a point cloud density ≥ 0.95.

[0088] Model optimization: An early stopping strategy is adopted during training. If the validation set loss does not decrease for 5 consecutive epochs, training is stopped and the optimal model parameters are saved. At the same time, gradient pruning (pruning threshold is 1.0) is used to avoid gradient explosion and improve model stability.

[0089] 3. Model Application and Data Transfer: The preprocessed dual-view images from S2 are compared... Input a pre-trained DUSt3R model, and the model outputs a dense point cloud through forward propagation. and camera pose This output data will serve as the core input to the S4 coordinate mapping model, providing a foundation for converting point clouds from pixel coordinates to geodetic latitude and longitude coordinates.

[0090] S4. Construction of a Three-Layer Coordinate Mapping Model and Point Cloud Coordinate Transformation

[0091] The core objective of this step is to construct a three-layer mapping relationship: "image pixels - gate chamber coordinate system - geodetic latitude and longitude coordinate system." This addresses the issue that the point cloud output by DUSt3R exists only in a pixel coordinate system and cannot be correlated with real-world geographic space. Through the calibration of a small number of ground control points, precise geolocation of the 3D model on the digital twin platform is achieved. The coordinate mapping and 3D mesh generation process is as follows: Figure 4 As shown.

[0092] 1. Ground Control Point Acquisition and Calibration: Eight ground control points (GCPs) were evenly distributed throughout the lock chamber area. The specific locations were: upper left corner of the lock chamber entrance, upper right corner of the entrance, left side of the middle section, right side of the middle section, upper left corner of the exit, upper right corner of the exit, starting point of the lock chamber centerline, and ending point of the lock chamber centerline, ensuring that the control points covered the entire navigation area of ​​the lock chamber. A GNSS receiver (model: Trimble R12i, positioning accuracy ±0.01m) was used to measure the three coordinate systems of each control point.

[0093] Pixel coordinate system coordinates The coordinates were obtained by manually marking the control point positions in the dual-view images acquired by S1. Subpixel-level precision marking tools were used to ensure that the coordinate error was ≤1 pixel.

[0094] Lock chamber coordinate system coordinates The gate chamber coordinate system is defined as a right-handed coordinate system, with the origin at the lower left corner of the gate chamber (entrance end). The axis runs along the length of the lock chamber (in the direction of navigation). The axis runs along the width of the lock chamber (perpendicular to the navigation direction). The axis is perpendicular to the water surface and pointing upwards (in the elevation direction). The three-dimensional coordinates of the control point are measured using a total station (model: Leica TS60).

[0095] Geodetic latitude and longitude coordinate system coordinates The WGS-84 coordinate system is used, and the coordinates are obtained through the static measurement mode of a GNSS receiver. The measurement time is ≥30 minutes to ensure that the positioning accuracy meets the requirements.

[0096] 2. Image Pixel-Gate Chamber Coordinate System Mapping Model (M1) Construction: A 4×4 affine mapping model is used to establish the relationship between pixel coordinates and gate chamber coordinates. Affine mapping can effectively preserve the parallelism and proportional relationship of the graphics and is suitable for linear transformation from planar to three-dimensional space. The formula is:

[0097] ;

[0098] in It is a 3×4 affine mapping matrix (the last row is fixed as [0,0,0,1]). For pixel coordinates, The coordinates of the gate chamber are given.

[0099] Solve using the least squares method The parameters are given, and the objective function is:

[0100] ;

[0101] The solution process is achieved through matrix operations: the coordinates of the eight control points are substituted into the formula to construct an overdetermined system of equations. (in The coefficient matrix, Let be the parameter vector to be determined. (where the observed values ​​are vectors), the least-squares solution of the system of equations is obtained by using singular value decomposition (SVD). Specific parameters (example parameters:) , , , The remaining parameters are obtained through solving. The core contribution of this model is to transform abstract pixel coordinates into physically meaningful gate chamber coordinates, laying the foundation for subsequent association with the geodetic coordinate system.

[0102] 3. Construction of the Lock Chamber Coordinate System-Geodetic Latitude and Longitude Coordinate System Mapping Model (M2): A 4×4 affine mapping model is also used to realize the transformation from lock chamber coordinates to geodetic latitude and longitude coordinates. The formula is:

[0103] ;

[0104] in For the affine mapping matrix, The coordinates are geodetic latitude and longitude (WGS-84).

[0105] Solve using the same least squares method as M1. The parameters and objective function are:

[0106] ;

[0107] After solving, we get Parameters (Example parameters:) , The remaining parameters are obtained through solving. The core function of this model is to establish the relationship between the local coordinate system (gate chamber) and the global coordinate system (geodetic latitude and longitude), enabling the point cloud to have geographic positioning capabilities.

[0108] 4. Model Interaction Verification: To ensure mapping accuracy, the pixel coordinates of eight ground control points were substituted into the three-layer mapping model. The calculated geodetic coordinates were compared with the measured values. The error thresholds were set as follows: latitude error ≤ 0.0001°, longitude error ≤ 0.0001°, and elevation error ≤ 0.1m. If the errors of all control points meet the requirements, the mapping model is valid; if there are control points with errors exceeding the limits, the control point labels or measurement data need to be rechecked, anomalies eliminated, and the model parameters recalculated until verification is successful.

[0109] 5. Point cloud coordinate transformation execution:

[0110] Pixel coordinate system → Gate chamber coordinate system: Convert the point cloud output by S3 Convert to homogeneous coordinates Substitute Calculate the coordinates of the gate chamber The formula is:

[0111] ;

[0112] Lock chamber coordinate system → Geodetic latitude and longitude coordinate system: Convert to homogeneous coordinates Substitute Calculate geodetic coordinates The formula is:

[0113] ;

[0114] Output: Dense point cloud in geodetic latitude and longitude coordinate system The point cloud already possesses accurate geolocation information and will be fed into S5 for the generation of a 3D mesh model.

[0115] S5. Optimization of 3D Mesh Model Generation and Geolocation Integration

[0116] This step generates a high-precision 3D mesh model of the ship based on the geodetic coordinate point cloud output by S4, and deeply integrates it with geolocation information. At the same time, the model's accuracy and positioning are improved through algorithm interaction optimization, ensuring that the model can realistically reproduce the ship's spatial position and shape in the digital twin platform.

[0117] 1. Point cloud preprocessing optimization:

[0118] Statistical filtering for noise reduction: The point cloud output by S4 may contain a small number of outliers (such as isolated points caused by measurement errors), which are removed using a statistical filtering algorithm. For each point in the point cloud, the average distance of its 50 neighboring points is calculated. and standard deviation If the average distance between the point and its neighborhood is greater than If an outlier is detected, it is identified as an anomaly and removed. This operation effectively cleans up point cloud data and prevents outliers from affecting the surface smoothness of the mesh model.

[0119] Voxel mesh downsampling: To improve the efficiency of subsequent mesh generation while preserving key features of the point cloud, voxel mesh downsampling is adopted. The voxel size is set to 0.05m × 0.05m × 0.05m (determined according to the ship's dimensional accuracy requirements). For each voxel, the coordinates of the center point of all points within the voxel are calculated, and this center point is retained as the downsampled point, ultimately yielding a simplified point cloud. (The number of point clouds is about 30% of the original), reducing computational complexity while ensuring model accuracy.

[0120] 2. 3D Mesh Model Generation: A Poisson surface reconstruction algorithm is used to generate a triangular mesh model. This algorithm can construct a smooth and continuous surface model from a dense point cloud, and is suitable for the reconstruction of complex curved objects such as ships. The specific steps are as follows:

[0121] Normal vector estimation: for For each point, select 30 neighboring points to fit a plane, and solve for the normal vector using the plane equation. To ensure that the normal vectors are aligned (all pointing outwards from the ship), a direction adjustment method based on principal component analysis (PCA) is used to make the normal vectors of all points align with the direction of the outward normal to the ship's surface.

[0122] Poisson equation construction: Constructing implicit functions based on point clouds and normal vectors The gradient of the implicit function is in the same direction as the normal vector and satisfies the Poisson equation. ,in For the Laplace operator, For divergence operators, Let be the set of point cloud normal vectors. By solving the Poisson equation, we obtain an implicit function that describes the ship's surface. .

[0123] Isosurface extraction: Extraction using the Moving Cubes algorithm. The isosurfaces, which represent the surface profile of the ship, are then used to generate a triangular mesh model. ,in For the set of vertices, This is a set of triangular faces (the number of faces is approximately twice the number of vertices). The quality of the triangular faces in the mesh model must meet the following requirements: minimum interior angle ≥ 30°, maximum interior angle ≤ 120°, to avoid elongated faces affecting the model's accuracy.

[0124] 3. Geolocation Fusion: Integrating grid models coordinates of each vertex The fusion formula is as follows: (Associate with the geographic coordinate system (WGS-84) of the digital twin platform.) ,in The geographic reference parameters for the digital twin platform are as follows (projection method: UTM, central meridian: 117°E, elevation datum: 1985 National Elevation Datum). During the fusion process, the model vertex coordinates are converted into a platform-compatible geographic coordinate format through the geographic registration interface provided by the platform, ensuring that the model's position on the platform is completely consistent with the ship's position in reality.

[0125] 4. Algorithm interaction optimization mechanism:

[0126] Mesh accuracy feedback optimization: Extracting key dimensional parameters of the mesh model (ship length) , ship width Draft ), and the point cloud size parameters output by DUSt3R in S3 ( , , Compare and calculate the dimensional error. .like Then adjust the resolution parameters of the Poisson reconstruction (increase the reconstruction depth by 1 level, from level 10 to level 11), and regenerate the mesh model until the size error is ≤3%.

[0127] Positioning accuracy feedback optimization: Verification using ground control points on the digital twin platform (corresponding one-to-one with S4 ground control points). The positioning error is calculated by determining the distance between the vertex coordinates corresponding to the control point in the model and the coordinates of the control point in the platform. ,in (Latitude distance coefficient) (Distance coefficient in longitude direction). If Then, the gradient descent method is used to optimize the coordinate mapping model. and Set the parameters (learning rate = 0.001, iteration count = 10), re-execute the point cloud transformation of S4 and the mesh generation of this step until the positioning error is ≤ 0.5m.

[0128] 5. Output: Optimized high-precision geolocation 3D mesh model The mesh model has ≥100,000 triangular faces and ≥50,000 vertices, which can clearly reproduce details such as the ship's hull structure, mast, and cargo loading. It also has accurate geolocation information, providing a high-quality model for error correction of S6 and integration with the digital twin platform of S7.

[0129] S6. Error Correction and Performance Verification

[0130] This step is a quality control step for the 3D reconstruction results. By quantitatively evaluating the size measurement error, positioning error and real-time performance, it ensures that the reconstruction results meet the actual application requirements of lock scheduling (size measurement error ≤ 4.8%, positioning error < 1m, single ship reconstruction time ≤ 30 seconds), and optimizes the model performance through a correction mechanism.

[0131] 1. Calculation and correction of dimensional measurement errors:

[0132] Actual dimensions were obtained using a laser rangefinder (model: Leica D510, measurement accuracy ±0.005m) to measure the actual dimensions of the vessel. Measurement points were selected as follows: Length: The straight-line distance between the foremost point of the bow and the aftmost point of the stern (along the navigation direction) was measured, and the average value of three measurement sections (bow, midships, and stern) was taken. Breadth: The straight-line distance between the widest points on both port and starboard sides (perpendicular to the navigation direction) was measured, and the average value of three measurement sections was taken. Draft: The distance between the freeboard deck and the water surface was measured and calculated using the vessel's draft depth data (obtained from the AIS system). The formula is as follows: ,in For ship depth, For the freeboard height, each measurement point is measured three times and the average value is taken to ensure the accuracy of the actual dimensional data.

[0133] Reconstruction Dimension Calculation: Mesh Model Output from S5 Extract the corresponding size parameters, ship length. The maximum distance between the front and rear vertices of the model (along the X-axis), and the ship's width. The maximum distance between the port and starboard vertices of the model (along the Y-axis), and the draft. This is the difference between the elevation of the bottom vertex of the model and the water surface elevation (the water surface elevation is obtained from the water level sensor).

[0134] Error Calculation and Correction: The formula for calculating dimensional measurement error is as follows: .like If the feature matching threshold of the DUSt3R model is adjusted (±0.05), the weights of the control points of the coordinate mapping model are adjusted (the weights of control points with large errors are increased to 1.2 times), and steps S3-S5 are re-executed until the dimensional measurement error is ≤4.8%.

[0135] 2. Positioning error calculation and correction:

[0136] Actual positioning acquisition: The actual geodetic coordinates of the ship's center are obtained through the ship's onboard AIS equipment (positioning accuracy ±1m). The AIS data sampling frequency was set to 1 time / second, and 10 sets of data were collected and the average value was taken as the ship's true positioning coordinates.

[0137] Reconstruction and positioning calculation: Extracting the mesh model center coordinates The calculation method is the arithmetic mean of the coordinates of all vertices, and the formula is: , ,in This represents the number of vertices in the model.

[0138] Error Calculation and Correction: The formula for calculating positioning error is as follows: .like Then optimize the coordinate mapping model. Set the parameters (learning rate = 0.001, number of iterations = 10), and re-execute steps S4-S5 until the positioning error is <1m.

[0139] 3. Real-time Verification: The entire process time from image acquisition in S1 to the final model output in S6 was recorded. The test hardware environment was: CPU Intel i9-13900K (24 cores, 32 threads), GPU NVIDIA RTX 4090 (24GB VRAM), RAM 64GB (DDR5 6400MHz), and storage NVMe SSD (2TB). Ten different types of ships were selected as test samples, and the reconstruction time for each ship was recorded, requiring an average reconstruction time of ≤30 seconds per ship. If the time exceeded the threshold, the voxel size of the point cloud downsampling was optimized (adjusted to 0.06m×0.06m×0.06m), and the feature extraction network of the DUSt3R model was simplified (reducing one Transformer encoder layer) to improve processing speed while maintaining accuracy.

[0140] 4. Final Output: After error correction and performance verification, a high-precision 3D mesh model of the ship that meets the requirements is output. It includes model files (in .obj format, supporting import from mainstream 3D software and digital twin platforms) and geolocation information files (in .json format, recording model center coordinates, vertex coordinate list, and coordinate system parameters), preparing for integration with the S7 digital twin platform.

[0141] S7. Digital Twin Platform Integration and Application

[0142] This step, as the final implementation of the entire technical solution, integrates the verified 3D mesh model into the digital twin platform, enabling real-time visualization of the model and linkage with business applications, providing intuitive and accurate digital support for lock scheduling and safety supervision.

[0143] 1. Model Format Conversion and Adaptation: Models output in .obj format from S6 need to be converted to .glTF format (an efficient 3D model transfer format that fully preserves geometry, materials, animation, and other information) supported by the digital twin platform. The conversion tool used is Blender 3.6, and the steps are as follows: Import the .obj model → Remove redundant vertices and faces → Optimize material textures (compress textures to Basis Universal format to reduce storage usage) → Export to .glTF 2.0 format. During the conversion process, ensure that the model's geometry, vertex coordinates, and texture information are not distorted, and keep the model file size under 50MB to meet the platform's real-time loading requirements.

[0144] 2. Platform Integration and Interface Interface: Model import is achieved through the RESTful API interface provided by the digital twin platform. The integration formula is as follows: ( =Ship model layer), where The "Ship Model Layer" is a platform-specific layer (with higher priority than the background layer to ensure clear model visibility) for obtaining the platform access token via API → uploading the model file and geographic information file → the platform parses the file and completes model registration → returns the integration result (success / failure and reason). During integration, the platform automatically matches the model's geographic coordinates with its own geographic scene, achieving precise overlay of the model at the corresponding location in the lock chamber.

[0145] 3. Real-time Linkage and Data Update: Establish a linkage mechanism between the model and real-time lock data. Through the platform's data subscription interface, acquire real-time dynamic data of the vessel (speed, heading, obtained from the AIS system) and lock chamber environmental data (water level, flow velocity, obtained from the sensor system), driving the model to update its attitude and position in real time on the platform. The linkage formula is as follows: ,in For timestamps, The data represents the ship's dynamics at time t. Here are the environmental data for the gate chamber at time t. This represents the ship model at time t in the platform. The data update frequency is set to once per second to ensure real-time synchronization between the model state and the actual ship state.

[0146] 4. Multi-dimensional application demonstration:

[0147] 3D visualization display: Supports interactive operation of the model, including rotation (mouse drag), scaling (scroll wheel zoom), and translation (hold down the middle mouse button and drag), while displaying key information of the ship (ship name, tonnage, cargo type, draft, and positioning coordinates). The information is overlaid on the model and updates synchronously as the model moves.

[0148] Data overlay analysis: The model is overlaid with the ship's lock passage status (waiting to lock, entering the lock, exiting the lock) and safety warning information (such as over-height and over-draft alarms). Different statuses are marked with different colors (waiting to lock: blue, entering the lock: yellow, exiting the lock: green, alarm: red) to intuitively present the entire process of ship lock passage.

[0149] Dispatch Auxiliary Support: Based on the space occupancy information of the model, the lock chamber queuing scheme is simulated in the platform. By dragging the model, the position of the ship in the lock chamber is adjusted, and the lock chamber space utilization rate is calculated. This provides a visual reference for dispatchers to formulate the optimal queuing scheme. The queuing scheme can be exported in Excel format, which includes parameters such as ship number, queuing position, and lock chamber utilization rate.

[0150] In some embodiments, a sparse coding-guided local feature enhancement algorithm (SC-LFE) is introduced, with the core objective of refining the local features of the ship. Through the collaborative design of three core modules—dictionary learning, sparse representation, and feature fusion—the feature extraction of DUSt3R is optimized. The specific construction process is as follows:

[0151] 1. Ship Feature Dictionary Construction Module: The core of this module is to learn a sparse dictionary adapted to ship features in the lock scenario, which is used for subsequent sparse representation and reconstruction of features. First, preprocessed dual-view images of 500 different types of ships (including cargo ships, container ships, dangerous goods ships, passenger ships, and engineering ships) are selected from the training set. For each ship, its key local feature regions (including hull edges, masts, load areas, bow and stern corners, deck texture blocks, etc.) are cropped. 16×16 pixel feature block samples are extracted from each local region, ultimately constructing a dictionary training sample set containing 100,000 feature blocks. Subsequently, the K-SVD algorithm is used to learn the dictionary from the sample set and initialize the dictionary. For random orthogonal matrices (number of atoms in the dictionary) Initially set to 1024, dictionary optimization is achieved through iterative updates: in each iteration, sparse coefficients are calculated for each feature block sample, and then dictionary atoms are updated based on the sparse coefficients until the dictionary converges (the convergence condition is that the change in dictionary reconstruction error between two adjacent iterations is ≤1e-5), ultimately yielding a ship-specific sparse dictionary. ( The value can be adjusted to 1024-2048 depending on the complexity of the ship's characteristics.

[0152] 2. Sparse Representation and Reconstruction Module: This module is used for sparse encoding and enhanced reconstruction of the deep features output by DUSt3R. First, the deep feature map extracted by DUSt3R is... ( (Corresponding to dual views) The clipping is performed in 16×16 pixel increments to obtain a set of non-overlapping local feature blocks. For each feature block The problem of constructing sparse representation optimization: ,in This is a sparsity regularization parameter (used to balance reconstruction accuracy and sparsity, with a value range of 0.01-0.05). This represents the sparse coefficient vector corresponding to the feature block. The optimization problem is solved using the Iterative Soft Thresholding Algorithm (ISTA). After obtaining the sparse coefficients, the algorithm then... Enhanced reconstruction of feature blocks is achieved, which improves the response intensity of key structural features of the ship while suppressing background interference features.

[0153] 3. Feature Fusion Module: To avoid losing global information of the original features during reconstruction, a weighted fusion mechanism is constructed to fuse the original feature blocks and the reconstructed feature blocks. The fusion formula is: ,in This is the fusion weight (initially set to 0.7). To adapt to the characteristics of different feature blocks, An adaptive adjustment strategy is adopted: the response intensity of the feature block is calculated. ,when When the response intensity is below a preset threshold (the average of the response intensities of all feature blocks), it indicates that the feature block contains relatively little effective information. Adjusted to 0.8-0.9 to enhance the contribution of reconstructed features; when When it is above the threshold, keep The original features and reconstructed features are balanced. Finally, the fused feature blocks are stitched together in the original cropping order to obtain a complete enhanced deep feature map. .

[0154] In some embodiments, the training of the SC-LFE algorithm is performed in conjunction with the training of the DUSt3R model. The dictionary parameters and fusion weights are optimized through an end-to-end training approach. The specific training process is as follows:

[0155] 1. Training Dataset Construction: The training dataset of the DUSt3R model (containing 5000 pairs of dual-view images of 500 ships and corresponding high-precision 3D point cloud labels) is reused and expanded: For each pair of dual-view images, key local feature regions of the ship (bow, stern, mast, and load area) are manually labeled to construct the dictionary training sample set and feature supervision during the training process.

[0156] 2. Loss Function Design: To ensure that the enhanced features of SC-LFE can improve the accuracy of subsequent 3D reconstruction, a multi-objective loss function is designed: ,in:

[0157] For feature reconstruction loss, , This represents the number of feature blocks in a single feature map, used to ensure consistency between the reconstructed features and the original features;

[0158] For sparsity loss, This is used to constrain the sparsity of sparse coefficients and improve the discriminability of features;

[0159] For feature matching loss, , To effectively match feature logarithms, this is used to ensure the matching performance of enhanced features;

[0160] , The loss weights are determined through cross-validation.

[0161] 3. Training parameter configuration: The Adam optimizer is used to optimize the dictionary of SC-LFE. fusion weight and sparse regularization parameters Optimization was performed with an initial learning rate of 0.001 and a weight decay coefficient of 1e-4; the batch size was set to 8 (each batch containing 8 sets of dual-view feature maps), and the training epochs were 30; an early stopping strategy was adopted, whereby the total loss on the validation set was reduced. If there is no decrease after 5 consecutive epochs, stop training and save the optimal model parameters.

[0162] 4. Training Optimization Strategy: During training, a dynamic learning rate adjustment mechanism is adopted, with the learning rate decaying to 0.9 every 10 epochs. For updating dictionary atoms, a mini-batch K-SVD strategy is adopted, updating only the 20% of atoms with the lowest contribution in each iteration to improve training efficiency. At the same time, sparse coefficients are pruned, setting coefficients with absolute values ​​less than 1e-6 to 0 to enhance sparsity.

[0163] In some embodiments, the SC-LFE algorithm does not run independently, but interacts deeply with the three core modules of DUSt3R feature extraction, triangulation point cloud estimation, and coordinate mapping model construction. Through bidirectional transmission of data flow and control flow, it achieves a comprehensive improvement in the accuracy of 3D reconstruction. Specific interactive applications are as follows:

[0164] (a) Interaction with DUSt3R feature extraction

[0165] SC-LFE takes the feature output of DUSt3R as input and optimizes the feature extraction performance of DUSt3R through the enhanced features, forming a closed-loop interaction of "extraction-enhancement-feedback".

[0166] 1. Forward Data Flow: DUSt3R extracts deep features from dual views through a multi-layer Transformer encoder. , The input is then fed into the SC-LFE algorithm, where it undergoes dictionary sparse representation, feature reconstruction, and weighted fusion to output enhanced features. , This enhancement feature strengthens the characteristic response of critical ship structures and suppresses interference features such as water surface reflection and floating objects.

[0167] 2. Reverse Feedback Flow: This refers to the sparse coefficients output by the SC-LFE. As a feature importance indicator, it is fed back to the multi-head self-attention layer of DUSt3R: for sparse coefficients For feature regions with larger norms (i.e., key structural regions of the ship), increase the attention weight of the self-attention layer, so that DUSt3R focuses more on feature extraction from these regions during subsequent training or inference; for sparse coefficients... For interference regions with smaller norms, reduce the attention weight and minimize the impact of interference features.

[0168] In some embodiments, the weight adjustment of the back feedback adopts a dynamic update method. Every 10 sets of dual-view images are processed, the initial weight matrix of the DUSt3R self-attention layer is updated based on the sparse coefficient statistics of SC-LFE, so that the interaction between the two can be adapted to the feature distribution of different types of ships.

[0169] (ii) Interaction with the triangulation algorithm

[0170] SC-LFE optimizes the point cloud estimation accuracy of the triangulation algorithm through confidence quantization and weighted solution. The interactive process runs through the entire workflow of "feature matching - coordinate solution".

[0171] 1. Confidence quantification propagation: SC-LFE for effective matching feature pairs Based on its corresponding sparsity coefficient , Calculate the matching confidence score , The closer the value is to 1, the higher the reliability of the match for that feature pair. It is passed as a weighting factor to the triangulation algorithm.

[0172] 2. Weighted triangulation solution: The triangulation algorithm is based on the epipolar geometric constraints of the two views to construct a system of linear equations. ( For the projection matrix, For three-dimensional point coordinates, (projected coordinates of feature points), and the confidence level passed by SC-LFE. By incorporating the objective function, we obtain a weighted optimization problem: The problem is solved using weighted least squares, which gives higher weight to high-confidence feature pairs in solving the 3D point coordinates and suppresses point cloud noise caused by false matches due to low confidence.

[0173] In some embodiments, if If the confidence threshold is low, the feature pair is directly removed to avoid affecting the point cloud accuracy. Simultaneously, the obtained 3D point cloud coordinates are fed back to SC-LFE to adjust the sparsity regularization parameters. If the average error of the point cloud is large, then increase... (Improve feature sparsity and reduce false matches), and vice versa. .

[0174] (III) Interaction with the coordinate mapping model

[0175] SC-LFE optimizes the transformation accuracy of the coordinate mapping model through key region localization and dynamic weight allocation, realizing the interactive logic of "feature enhancement - region localization - weight adjustment":

[0176] 1. Key Area Location Transfer: Based on the response intensity of sparse coefficients, SC-LFE locates the point cloud subsets corresponding to key areas of the ship (bow, stern, and load area). (3D points corresponding to feature blocks with response intensity 1.5 times higher than the mean), and The spatial coordinate range is transferred to the coordinate mapping model.

[0177] 2. Dynamic adjustment of control point weights: The coordinate mapping model is used in constructing the image pixel-gatehouse coordinate system mapping model ( ) and the mapping model of the gate chamber coordinate system-geodetic latitude and longitude coordinate system ( When, according to the SC-LFE transmitted Spatial scope: Identify the corresponding physical key areas within the gate chamber and assign dynamic weights to the ground control points in that area. ,in The initial weights for the control points are uniformly distributed as 1. for The corresponding mean of the sparse coefficients. Dynamic weighting allows control points in key areas to occupy a higher proportion in the mapping matrix solution, improving the coordinate transformation accuracy of key areas of the ship.

[0178] In some embodiments, after the coordinate mapping model is solved, the calculation... If the coordinate transformation error exceeds a preset threshold (0.1m), it is fed back to SC-LFE to adjust the feature fusion weights. (Increase to 0.8-0.9), then repeat feature enhancement and coordinate transformation until the error meets the requirements.

[0179] The embodiments described above are for illustrative purposes only and are not intended to limit the invention. Therefore, any changes in numerical values ​​or substitutions of equivalent elements should still fall within the scope of this invention.

[0180] The above detailed description will enable those skilled in the art to understand that the present invention can indeed achieve the aforementioned objectives and has complied with the provisions of the Patent Law.

Claims

1. A calibration-free 3D reconstruction method for ship locks based on dual views and DUSt3R, characterized in that, include: S1. Simultaneously acquire front and side view images of navigating vessels to form a dual-view image pair; S2. The dual-view image pair is sequentially subjected to noise suppression, reflection suppression and distortion correction to obtain a preprocessed dual-view image pair; S3. Input the preprocessed dual-view image into the trained DUSt3R algorithm model based on the Transformer architecture, extract the deep features of the dual-view image, perform feature matching using a feature matching mechanism, and estimate the dense point cloud based on the effective matching feature pairs obtained by feature matching through a triangulation algorithm. S4. Set up multiple ground control points in the gate chamber area, obtain the pixel coordinates, gate chamber coordinates and geodetic latitude and longitude coordinates of each control point, construct an image pixel-gate chamber coordinate system mapping model and a gate chamber coordinate system-geodetic latitude and longitude coordinate system mapping model, and convert the dense point cloud into a dense point cloud in the geodetic latitude and longitude coordinate system in sequence through the two mapping models. S5. Denoise and downsample the dense point cloud in the geodetic latitude and longitude coordinate system. Based on the processed point cloud, generate a three-dimensional mesh model through a surface reconstruction algorithm. Associate the three-dimensional mesh model with the geographic coordinate system to achieve geographic positioning fusion and obtain the geographically positioned three-dimensional mesh model. S6. Obtain the actual dimensions and actual positioning coordinates of the ship, and calculate the size measurement error and positioning error of the three-dimensional mesh model after geolocation. If the error exceeds the corresponding preset error threshold, adjust the relevant algorithm parameters and repeat steps S3-S5 until the error meets the requirements, and obtain the three-dimensional mesh model after error correction.

2. The method for calibration-free 3D reconstruction of ship locks based on dual views and DUSt3R according to claim 1, characterized in that, Also includes: S7. Convert the error-corrected 3D mesh model into a format supported by the digital twin platform, import it through the platform interface and link it with the real-time data of the lock to realize the visualization and business application of the model.

3. The method for calibration-free 3D reconstruction of ship locks based on dual views and DUSt3R as described in claim 1, characterized in that, In S3, the feature extraction module of the DUST3R algorithm model adopts a multi-layer Transformer encoder. Each encoder layer includes a multi-head self-attention layer and a feedforward neural network layer. The multi-head self-attention layer captures the long-distance dependencies between image pixels in parallel through multiple sets of attention heads. The feedforward neural network layer performs dimensional transformation and enhancement on the extracted features through a non-linear activation function.

4. The method for calibration-free 3D reconstruction of ship locks based on dual views and DUSt3R according to claim 1, characterized in that, In S3, the feature matching mechanism uses cross-correlation attention to calculate the matching similarity of deep features in the two views, and filters effective matching feature pairs through a dynamic matching degree threshold. The dynamic matching degree threshold is adaptively adjusted according to the illumination changes and ship attitude differences in the two view images. At the same time, false matching features are eliminated through neighborhood feature consistency verification.

5. The method for calibration-free 3D reconstruction of ship locks based on dual views and DUSt3R according to claim 1, characterized in that, In S4, both the image pixel-gate chamber coordinate system mapping model and the gate chamber coordinate system-geodetic latitude and longitude coordinate system mapping model adopt affine mapping models. The mapping matrix parameters are solved by the least squares method. During the solution process, a weight allocation strategy for control point coordinates is introduced to give higher weights to the control points in the key areas of the gate chamber, realize the conversion from gate chamber coordinates to geodesic latitude and longitude coordinates, establish the association between gate chamber coordinates and geodesic latitude and longitude coordinates, and enable the point cloud to have geographic positioning capabilities.

6. The method for calibration-free 3D reconstruction of ship locks based on dual views and DUSt3R according to claim 1, characterized in that, In S5, the denoising adopts a statistical filtering algorithm. By calculating the average distance and standard deviation of a preset number of neighboring points for each point, points that exceed the range of the average distance and the preset multiple of the standard deviation are identified as outliers and removed. The downsampling process adopts a voxel grid downsampling algorithm. The voxel size is dynamically adapted according to the density characteristics of the dense point cloud and the accuracy requirements of the three-dimensional mesh model.

7. The method for calibration-free 3D reconstruction of ship locks based on dual views and DUSt3R according to claim 1, characterized in that, In S3, the triangulation algorithm is based on the epipolar geometric constraints of the dual-view system. It constructs a system of linear equations to solve the three-dimensional spatial coordinates by projecting the corresponding feature points in the coordinate systems of the two cameras. During the solution process, a robust estimation strategy is introduced to suppress the influence of matching error on the solution results of the three-dimensional point cloud coordinates.

8. The method for calibration-free 3D reconstruction of ship locks based on dual views and DUSt3R according to claim 1, characterized in that, In S6, the relevant algorithm parameters include the number of feature extraction network layers of the DUSt3R algorithm model, the feature matching threshold, the robust estimation coefficients of the triangulation algorithm, the control point weights of the coordinate mapping model, and the resolution parameters of the surface reconstruction algorithm. The parameter adjustment is optimized in a directional manner according to the distribution characteristics of the size measurement error and the positioning error until the error meets the preset requirements.

9. The method for calibration-free 3D reconstruction of ship locks based on dual views and DUSt3R according to claim 1, characterized in that, In S5, the surface reconstruction algorithm adopts the Poisson surface reconstruction algorithm. During the reconstruction process, the normal vector of the point cloud is estimated by fitting the plane through the neighborhood points and the direction of the normal vector is unified. Then, an implicit function is constructed based on the point cloud and the normal vector. A triangular mesh model is generated by extracting the isosurface of the preset implicit function value. The smoothness parameter of the implicit function is adaptively set according to the density of the point cloud.

10. The method for calibration-free 3D reconstruction of ship locks based on dual views and DUSt3R according to claim 1, characterized in that, In S2, the noise suppression uses a Gaussian filtering algorithm, and the size of the Gaussian kernel function is dynamically selected according to the image noise intensity; the reflection suppression uses the Otsu method to solve the adaptive threshold segmentation of the reflective area, performs brightness compression processing on the reflective area, and performs histogram equalization on the non-reflective area to enhance feature contrast; the distortion correction uses a quadratic polynomial distortion model, obtains the distortion coefficients through a standard calibration template, and accurately corrects the radial distortion of the image.

Images