Ship water gauge dynamic calibration identification system and method based on multi-modal data fusion

Abstract

The application belongs to the field of image processing, and provides a ship water gauge dynamic calibration identification system and method based on multi-modal data fusion. The method comprises the following steps: collecting visual image data of a water gauge scale area; constructing a virtual space model of the water gauge scale area based on the visual image data, and optimizing the virtual space model in combination with a pre-set physical constraint; performing dynamic semantic segmentation on the virtual space model, dividing the water gauge scale area and interference objects, and calculating a first water level line of a target ship in real time based on the water gauge scale area; acquiring real-time measurement data of the target ship and a water area where the target ship is located; performing multi-modal fusion on the real-time measurement data through a graph neural network, constructing a dynamic calibration model based on the fusion result, and predicting a real-time position relationship between the target ship and the water area; and performing dynamic calibration on the first water level line based on the real-time position relationship, and obtaining a second water level line. The error of the visual recognition result is dynamically corrected, and the measurement accuracy and efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a dynamic calibration and identification system and method for ship draft gauge based on multimodal data fusion. Background Technology

[0002] A water gauge is a scale device that directly observes changes in water level. It reflects the height difference of the water surface relative to a reference point through scale markings and is widely used in water conservancy projects, hydrological monitoring, and waterway management.

[0003] For ship draft gauges, prolonged overloading or improper ballast can cause hull distortion (such as midship concavity or localized bulge), resulting in a misalignment between the draft gauge markings and the actual draft. For example, a ship's midship deformation caused a 12cm difference in draft between port and starboard sides. Furthermore, rust or wear on the hull plates can blur or shift the draft gauge markings, affecting the accuracy of readings.

[0004] Therefore, in view of the above-mentioned technical problems, it is urgent to design a brand-new technical solution to solve at least one of the above-mentioned technical problems. Summary of the Invention

[0005] The main objective of this application is to provide a dynamic calibration and identification system and method for ship draft gauges based on multimodal data fusion, aiming to solve the technical problem of decreased draft gauge accuracy caused by hull deformation or corrosion in related technologies.

[0006] In a first aspect, embodiments of this application provide a method for dynamic calibration and identification of ship draft gauges based on multimodal data fusion, including:

[0007] The visual image module is used to collect visual image data of the water gauge scale area;

[0008] Based on the visual image data, a virtual spatial model of the water gauge scale area is constructed through a neural radiation field, and the virtual spatial model is optimized in combination with pre-set physical constraints; wherein, the physical constraints include at least: ship hull operating status, ship weight distribution, cargo type, and cargo volume.

[0009] Dynamic semantic segmentation is performed on the optimized virtual space model to divide the water gauge scale area and interference objects, and the first water level line of the target ship is calculated in real time based on the water gauge scale area.

[0010] Real-time measurement data of the target vessel and the water area it is in are obtained through laser rangefinders, inertial measurement modules, and water level gauges.

[0011] The real-time measurement data is fused using a graph neural network to perform multimodal fusion. Based on the fusion results, a dynamic calibration model matching the target ship is constructed to predict the real-time positional relationship between the target ship and the water area. The dynamic calibration model incorporates the physical prior conditions of the ship's hydrostatic curve equation.

[0012] Based on the real-time positional relationship, the first water level line in the virtual space model is dynamically calibrated to obtain the second water level line of the target ship.

[0013] Secondly, embodiments of this application provide a ship draft dynamic calibration and identification system based on multimodal data fusion, comprising:

[0014] The acquisition module is used to acquire visual image data of the water level gauge area through the visual image module; and to obtain real-time measurement data of the target vessel and the water area it is in through the laser rangefinder, inertial measurement module and water level gauge.

[0015] The construction module is used to construct a virtual spatial model of the water gauge scale area based on the visual image data through a neural radiation field, and optimize the virtual spatial model in combination with pre-set physical constraints; wherein, the physical constraints include at least: ship hull operating state, ship weight distribution, cargo type, and cargo load.

[0016] The identification module is used to perform dynamic semantic segmentation on the optimized virtual space model, divide the water gauge scale area and interference objects, and calculate the first water level line of the target ship in real time based on the water gauge scale area.

[0017] The fusion module is used to perform multimodal fusion of the real-time measurement data through a graph neural network, and to construct a dynamic calibration model that matches the target ship based on the fusion result, predicting the real-time positional relationship between the target ship and the water area; the dynamic calibration model is embedded with the physical prior conditions of the ship's hydrostatic curve equation;

[0018] The calibration module is used to dynamically calibrate the first water level line in the virtual space model based on the real-time position relationship, so as to obtain the second water level line of the target ship.

[0019] Thirdly, embodiments of this application also provide a terminal device, which includes a processor and a memory for storing a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the dynamic calibration and identification method for ship draft based on multimodal data fusion as described in the first aspect or any embodiment of this application.

[0020] This application provides a system and method for dynamic calibration and identification of ship draft based on multimodal data fusion. In this method, visual image data of the water gauge scale area is acquired through a visual image module; based on the visual image data, a virtual spatial model of the water gauge scale area is constructed through a neural radiation field, and the virtual spatial model is optimized in combination with pre-set physical constraints; wherein, the physical constraints include at least: ship operating state, ship weight distribution, cargo type, and cargo load; the optimized virtual spatial model is dynamically semantically segmented to divide the water gauge scale area and interference objects, and the first water level line of the target ship is calculated in real time based on the water gauge scale area; real-time measurement data of the target ship and the water area it is in are acquired through a laser rangefinder, an inertial measurement module, and a water level gauge; the real-time measurement data is fused in a multimodal manner through a graph neural network, and a dynamic calibration model matching the target ship is constructed based on the fusion result to predict the real-time positional relationship between the target ship and the water area; the dynamic calibration model embeds the physical prior conditions of the ship's hydrostatic curve equation; based on the real-time positional relationship, the first water level line in the virtual spatial model is dynamically calibrated to obtain the second water level line of the target ship.

[0021] This method effectively overcomes the shortcomings of traditional pure visual recognition, which is prone to misjudgment due to environmental interference, by deeply fusing physical data such as laser ranging and inertial measurement with visual information. Simultaneously, it utilizes a dynamic calibration model embedded with the ship's hydrostatic curve equation to achieve real-time correction of water level lines under scenarios such as cargo changes. To improve accuracy, a high-precision virtual space model is constructed using neural radiation fields and physical constraints, and water level gauge information is accurately extracted through dynamic semantic segmentation. Real-time automated processing of multimodal data is achieved using graph neural networks, combined with intelligent prediction and dynamic calibration mechanisms, significantly improving water level measurement efficiency. Furthermore, generative adversarial networks are used to enhance model training, enabling the system to possess good robustness and environmental adaptability under different ship types, sea states, and lighting conditions. This comprehensive breakthrough overcomes traditional technical bottlenecks, providing a more reliable and efficient solution for ship water level measurement. Attached Figure Description

[0022] Figure 1 A flowchart illustrating a method for dynamic calibration and identification of ship draft based on multimodal data fusion, provided for an embodiment of this application;

[0023] Figure 2 A schematic diagram of the module structure of a ship draft dynamic calibration and identification system based on multimodal data fusion is provided for an embodiment of this application;

[0024] Figure 3 This is a schematic block diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0025] This application provides a system and method for dynamic calibration and identification of ship draft gauges based on multimodal data fusion. The method can be applied to a terminal device, such as a mobile terminal, mobile phone, virtual reality device, tablet computer, laptop computer, desktop computer, wearable device, or other electronic device. The terminal device can be a cloud server connected to a ship management system or a server cluster. The connection can be implemented through hardware circuitry or a communication module.

[0026] The following detailed description, in conjunction with the accompanying drawings, illustrates some embodiments of this application. Unless otherwise specified, the following embodiments and features can be combined with each other. Please refer to... Figure 1 , Figure 1 This is a flowchart illustrating a dynamic calibration and identification method for ship draft gauges based on multimodal data fusion, provided as an embodiment of this application.

[0027] like Figure 1 As shown, the ship draft dynamic calibration and identification method based on multimodal data fusion includes the following steps:

[0028] Step S101: Collect visual image data of the water level gauge scale area through the visual image module.

[0029] In this embodiment, visual image data refers to image information acquired by the visual image module for water level gauge identification. It is acquired via a high-definition camera (such as an industrial-grade area scan camera or a high-resolution surveillance camera) in the form of a video stream or single-frame image. The camera can be installed at a fixed location on the ship's deck to capture water level gauge images from a top-down or side-view angle; it can also be deployed at shore-based monitoring points to acquire ship water level gauge images from a distance, meeting monitoring needs in different scenarios. The image must completely cover the ship's water level gauge marking area, including the digital markings, marking lines, and a certain range of the ship's hull surface surrounding the water level gauge. Simultaneously, the waterline where the water level gauge meets the water surface is captured, providing crucial information for subsequent water level calculations.

[0030] To ensure recognition accuracy, the visual image data must have a high resolution to ensure that the details of the water level gauge are clearly distinguishable. Regarding frame rate, if real-time monitoring of dynamic scenes is required, a frame rate of 25 frames per second or higher is recommended to avoid motion blur. In addition, good color reproduction and a wide dynamic range are also necessary to adapt to different lighting conditions (such as direct sunlight and nighttime illumination).

[0031] The acquired raw images usually need to be preprocessed, including but not limited to denoising (removing noise and snow interference from the image), geometric correction (correcting image distortion caused by the shooting angle), brightness and contrast adjustment, etc., to improve image quality and lay the foundation for subsequent steps such as building a virtual space model based on neural radiation field and dynamic semantic segmentation.

[0032] Under varying lighting conditions, the visual image module may experience reduced clarity and color distortion in the data collected, particularly when faced with issues such as glare and shadows from direct sunlight or image blurring due to insufficient light at night. However, by employing a multimodal data fusion mechanism and incorporating physical draft data from a laser rangefinder, the system can directly avoid the interference of lighting on visual recognition, ensuring the accuracy of water level measurements. Simultaneously, the virtual space model constructed by combining neural radiation fields with physical constraints remains unaffected by lighting changes, continuously providing stable spatial information for the water level gauge scale. This, combined with dynamic semantic segmentation technology, allows for precise extraction of the water level gauge region. Furthermore, a generative adversarial network (GAN) pre-simulates the impact of different lighting scenarios on the data, and the trained dynamic calibration model exhibits strong robustness, ensuring stable and reliable measurement performance under various lighting conditions, including day-night cycles and changes in weather.

[0033] Step S102: Based on the visual image data, construct a virtual spatial model of the water level scale area through a neural radiation field, and optimize the virtual spatial model in combination with pre-set physical constraints.

[0034] In this embodiment of the application, the physical constraints include at least: the ship's operating state, the ship's weight distribution, the type of cargo, and the cargo volume.

[0035] Specifically, in the embodiments of this application, physical constraints such as the ship's operating status, ship's weight distribution, cargo type, and cargo volume provide accurate basis for the dynamic calibration and identification of the ship's draft gauge from different dimensions.

[0036] Ship operational status: This encompasses the ship's attitude information, including roll, pitch, and bow, as well as motion parameters such as speed and acceleration. Inertial Measurement Unit (IMU) data is collected in real-time to provide the system with real-time attitude information of the ship in three-dimensional space. When the ship tilts, the visual representation of the draft gauge readings is distorted due to perspective, making traditional purely visual recognition prone to errors. However, by incorporating constraints related to the ship's operational status, the draft gauge attitude in the virtual space model can be adjusted synchronously to ensure that the draft gauge readings are aligned with the actual ship attitude, eliminating waterline positioning deviations caused by ship movement.

[0037] Ship weight distribution: Closely related to the ship's structural design and cargo loading position, it determines the ship's stress equilibrium state. Different weight distributions will cause the ship to sink and tilt to varying degrees. Based on the ship's design drawings and real-time cargo loading conditions, the system calculates the ship's weight distribution, transforms it into constraints, and integrates it into a virtual space model. This ensures that the draft gauge position in the model matches the actual stress state of the ship, avoiding draft gauge reading errors caused by uneven weight distribution and ensuring that the measurement results conform to the ship's true mechanical equilibrium.

[0038] Cargo Type: Different types of cargo (such as bulk cargo, containers, liquid cargo, etc.) have different stacking characteristics and center of gravity distribution patterns. Bulk cargo may shift due to swaying during navigation, changing the ship's center of gravity; liquid cargo is subject to the free surface effect, affecting ship stability. Based on the characteristics of different cargo types, the system optimizes the virtual space model accordingly, adjusting the calculation methods for draft gauges and water levels. This allows the model to adapt to the impact of different cargo types on the ship's draft and attitude, improving the accuracy and reliability of identification.

[0039] Cargo load: Directly determines the ship's draft and is a key factor affecting draft readings. As cargo is loaded and unloaded, the ship's cargo load changes continuously, and the draft changes accordingly. The system acquires cargo load data in real time and, combined with the ship's hydrostatic curve equations, dynamically corrects the draft gauge position and draft in the virtual space model. This enables real-time updates of the waterline during cargo loading, ensuring that draft measurement results accurately and promptly reflect changes in the ship's cargo status.

[0040] For example, in step S102, based on the visual image data, a virtual spatial model of the water gauge scale region is constructed using a neural radiation field, and the virtual spatial model is optimized in conjunction with pre-set physical constraints, including:

[0041] Multi-view images from the visual image data are acquired, and image feature points are extracted using a feature extraction network. Sparse point cloud reconstruction is performed based on the feature points to generate an initial 3D point cloud model of the water gauge scale region. The initial 3D point cloud model is input into a neural radiation field model, and the neural radiation field is converted into a renderable virtual space model using volume rendering technology. Based on the pre-set physical constraints corresponding to the ship's operating state, ship's weight distribution, cargo type, and cargo volume, a comprehensive constraint loss function is constructed to optimize the parameters of the neural radiation field model. The optimized parameters are then re-inputted into the neural radiation field model to generate a virtual space model containing the water gauge scale region.

[0042] Specifically, in step S102, the process of constructing and optimizing the virtual spatial model of the water level scale area based on visual image data is a key step in achieving high-precision modeling through the integration of multiple technologies.

[0043] The visual image module acquires multi-view images covering information about the water level gauge area from different angles. Through feature extraction networks (such as classic algorithms like SIFT, SURF, and ORB, or deep learning-based models like SuperPoint and LoFTR), unique and stable feature points can be extracted from the images. These feature points contain crucial information such as the inflection points of the water level gauge lines and the edges of the digit contours, providing a foundation for subsequent 3D reconstruction. For example, the SuperPoint model uses a convolutional neural network to automatically learn feature points and their descriptors in the image, exhibiting higher accuracy and robustness compared to traditional methods in complex lighting and texture environments.

[0044] Based on the extracted feature points, feature point pairs in multi-view images are found using feature matching algorithms (such as brute-force matching and FLANN matching). Then, the coordinates of these feature points in three-dimensional space are calculated using the principle of triangulation, thereby generating an initial three-dimensional point cloud model of the water gauge scale area. This process converts two-dimensional image information into three-dimensional spatial information, initially restoring the geometry of the water gauge, but the point cloud is relatively sparse, containing only the spatial positions of the feature points.

[0045] An initial 3D point cloud model is input into a Neural Radiation Field (NeRF) model. NeRF learns the volume density and radiance (color) information of each point in the scene using a Multilayer Perceptron (MLP) to construct a continuous 3D scene representation. During training, by inputting the direction and position of light rays from different viewpoints, NeRF predicts the color and opacity at the intersections of light rays and the scene. This information is then accumulated using volumetric rendering technology to generate a renderable virtual space model. Volumetric rendering technology can simulate the propagation and absorption of light rays in the scene, ultimately outputting a realistic 3D model image that meticulously reproduces the texture and geometric details of the watermark scale.

[0046] For example, when constructing the comprehensive constraint loss function, physical constraints such as hull operating state, hull weight distribution, cargo type, and cargo volume can be incorporated. Specifically, using real-time attitude data such as hull roll, pitch, and bow angles collected by the inertial measurement unit (IMU), the difference between the hull attitude parameters in the model and the IMU measurements is used as a constraint term to ensure that the draft gauge reading in the model is consistent with the actual hull attitude, avoiding visual distortion of the reading due to hull tilt. The hull weight distribution is calculated based on the ship's design drawings and real-time cargo loading. Constraint terms reflecting the balance of gravity distribution are introduced into the comprehensive constraint loss function to ensure that the draft gauge position in the model matches the actual stress state of the hull, avoiding draft gauge reading errors caused by uneven gravity distribution. Considering the different impacts of different cargo types (such as bulk cargo, containerized cargo, and liquid cargo) on the ship's center of gravity and stability, constraint terms related to cargo type are added to the comprehensive constraint loss function. The model is optimized for the characteristics of different cargo types, adjusting the calculation methods of the draft gauge reading and waterline. By incorporating the ship's hydrostatic curve equations, the relationship between cargo load and draft is integrated into the comprehensive constraint loss function. The difference between the draft in the comprehensive constraint loss function and the theoretical draft corresponding to the actual cargo load is used to dynamically correct the position of the draft gauge and the draft in the virtual space model.

[0047] Therefore, the aforementioned physical constraints are transformed into corresponding loss terms, which are combined with the visual reconstruction loss of the neural radiation field model (i.e., the difference between the rendered image and the original visual image) to form a comprehensive constraint loss function. The parameters of the neural radiation field model are iteratively optimized using the backpropagation algorithm to minimize the comprehensive constraint loss function, thereby obtaining a virtual space model that conforms to physical laws and highly matches the actual scene.

[0048] Furthermore, in an optional example, the various physical constraints mentioned above can be transformed into corresponding loss terms through a comprehensive constraint loss function, which is then combined with the visual reconstruction loss of the neural radiation field model (i.e., the difference between the rendered image and the original visual image) to form a comprehensive constraint loss function. The parameters of the neural radiation field model are iteratively optimized using the backpropagation algorithm to minimize the comprehensive constraint loss function, thereby obtaining a virtual space model that conforms to physical laws and highly matches the actual scene.

[0049] Finally, the optimized neural radiation field model parameters were re-input into the model, and the final virtual space model was generated using volume rendering technology with the updated volume density and radiance information. This model not only accurately reproduced the geometric and texture details of the waterline scale area, but also fully considered the influence of physical factors such as the ship's operating state, gravity distribution, cargo type, and cargo volume, providing an accurate and reliable foundation for subsequent dynamic semantic segmentation and waterline calculation.

[0050] Step S103: Perform dynamic semantic segmentation on the optimized virtual space model to divide the water gauge scale area and interference objects, and calculate the first water level line of the target ship in real time based on the water gauge scale area.

[0051] For example, in step S103 above, the virtual space model containing the water gauge scale area is divided into three-dimensional grid units to obtain a semantic segmentation network containing multiple three-dimensional grid units; based on the spatiotemporal attention mechanism, the three-dimensional grid units in the semantic segmentation network are classified to identify the water gauge scale area, rust, water ripples, and hull; edge detection is performed on the segmented water gauge scale area to extract the scale line contour; according to the preset scale line spacing and scale standard, the height difference between adjacent scale lines is calculated, and the scale line contour is optimized using the height difference to obtain the three-dimensional position coordinates of the water gauge scale; combined with the transformation relationship between the image coordinate system and the world coordinate system, the three-dimensional position coordinates are converted into actual physical coordinates, and the first water level line is calculated in real time based on the physical coordinates of the water gauge scale.

[0052] Specifically, in step S103, dynamic semantic segmentation and the calculation of the first water level rely on 3D modeling, deep learning and geometric calculation to achieve accurate recognition.

[0053] First, the optimized virtual space model is divided into uniform 1cm×1cm×1cm cube grid units. The continuous model is discretized using the Marching Cubes algorithm or volumetric rendering sampling to reduce computational complexity. Each grid unit carries geometric coordinates, visual color, and volume density values, while also being associated with physical parameters such as ship attitude and gravity distribution. Second, a 3D semantic segmentation network such as U-Net3D or PointNet++ is constructed, embedding a spatiotemporal attention module. Spatially, a self-attention mechanism is used to capture the continuity of the scale lines, and temporally, RNNs or 3D convolutions are used to distinguish between static scales and dynamic interference, achieving accurate classification of water level markings, rust, water ripples, etc. Next, edge detection is performed on the segmented scale regions using the 3D Canny3D or 3D Sobel3D operator: spatial edge features of the scale lines are extracted through 3D gradient calculation to generate an initial contour point cloud. To address potential noise in point clouds, the standard size characteristics of the main tick marks (10cm spacing) and auxiliary tick marks (1cm spacing) are combined. First, a clustering algorithm (such as DBSCAN) is used to classify spatially adjacent points into potential tick mark groups. Then, a least squares method is used to fit a straight line or curve to each group of point clouds. Finally, noise points deviating from the fitted model are removed, resulting in accurate 3D coordinates of the tick marks. Canny3D is a 3D extension of the 2D Canny algorithm. It achieves edge detection through a series of steps: first, Gaussian filtering is applied to the volume data in 3D space to smooth it and reduce noise interference; then, the gradient magnitude and direction of each voxel are calculated using a 3D difference operator; subsequently, non-maximum suppression is performed on the gradient direction, retaining local maxima to refine the edges; finally, double threshold segmentation distinguishes strong and weak edges, retaining only weak edges connected to strong edges, ultimately obtaining continuous 3D edges. Sobel3D calculates the gradient of the volume data based on 3D differential convolution kernels. By applying the Sobel operator in the x, y, and z directions, it obtains the gradient magnitude and direction of each voxel in 3D space. Compared to Canny3D, Sobel3D is simpler to implement and faster in computation, but it is more sensitive to noise, and the edge extraction results may contain more redundant information. Finally, based on the camera calibration parameters, the world coordinate system and image coordinate system are transformed. The intersection of the waterline and the scale line in the image is located, projected back to the world coordinate system, and combined with laser ranging data or ship hydrostatic curves, the vertical coordinates of the waterline are calibrated to determine the physical location of the first waterline.

[0054] Thus, by leveraging 3D semantic segmentation to enhance resistance to rust and water ripple interference, utilizing spatiotemporal attention to track hull sway, and deeply integrating physical constraints to ensure that the results meet design specifications, a reliable foundation is laid for subsequent multimodal calibration.

[0055] Step S104: Acquire real-time measurement data of the target vessel and the water area it is in using a laser rangefinder, inertial measurement module, and water level gauge.

[0056] For example, in step S104 above, real-time measurement data of the target vessel and the water area it is in are obtained through a laser ranging sensor, an inertial measurement module, and a water level gauge, including:

[0057] The laser rangefinder sensor is initialized, and the measurement frequency and range are set. During the operation of the target vessel, the laser rangefinder sensor emits a laser beam into the water area and receives the reflected laser signal. The distance from the sensor to the water surface is calculated based on the flight time. The inertial measurement module collects the three-axis acceleration, angular velocity, and magnetic field strength data of the target vessel in real time. Noise is removed by a filtering algorithm, and the attitude angle of the vessel is calculated. A water level gauge is installed at a static reference point to measure the vertical distance between the reference point and the water surface in real time. The laser rangefinder data, inertial measurement data, and water level gauge data are timestamped to form real-time measurement data of the target vessel and the water area it is in.

[0058] Specifically, in step S104, the acquisition of real-time measurement data is the foundation of multimodal fusion. Its core lies in the collaborative acquisition of key physical information through a laser rangefinder, inertial measurement module, and water level gauge. During the startup phase, the laser rangefinder needs to be initialized, setting an appropriate measurement frequency (e.g., 10 times per second) and measurement range (covering the ship's maximum draft) according to the actual application scenario to ensure stable operation. During ship operation, the laser rangefinder continuously emits a laser beam into the water area. By capturing the flight time of the laser from emission to reflection back to the sensor, the vertical distance from the sensor to the water surface is accurately calculated. This data directly reflects the ship's current draft. The inertial measurement module collects the ship's three-axis acceleration, angular velocity, and magnetic field strength data in real time. Since the raw data often contains environmental noise and equipment errors, it needs to be processed using algorithms such as Kalman filtering or complementary filtering to effectively remove noise interference. Afterward, the ship's real-time roll, pitch, and bow angles are further calculated to accurately obtain the ship's attitude information. If a water level gauge is used, it must be installed at a static reference point unaffected by ship movement. The built-in level sensor measures the vertical distance between the reference point and the water surface in real time, providing a reference for water level calculation. Finally, to ensure consistency of multi-source data over time, timestamps accurate to milliseconds must be added to the laser ranging data, inertial measurement data, and water level gauge data. These data are then integrated into a complete real-time measurement dataset of the target ship and its surrounding water area, providing reliable physical data support for subsequent multimodal data fusion and dynamic calibration.

[0059] Step S105: The real-time measurement data is fused using a graph neural network to form a multimodal model. Based on the fusion result, a dynamic calibration model matching the target ship is constructed to predict the real-time positional relationship between the target ship and the water area.

[0060] In this embodiment, the dynamic calibration model incorporates the physical prior conditions of the ship's hydrostatic curve equation.

[0061] The purpose of physical prior conditions is to combine the hydrodynamic characteristics of a ship with real-time measurement data to improve the accuracy and reliability of water level monitoring. Specifically, the hydrostatic curve equation of a ship is a mathematical model that describes the relationship between parameters such as the ship's displacement volume, center of buoyancy, metacentric radius, and draft in still water. It is usually pre-constructed based on ship design drawings and hydrodynamic calculations, reflecting the ship's own physical properties and hydrodynamic laws.

[0062] The dynamic calibration model embeds these physical prior conditions into its model architecture in the form of equations. The hydrostatic curve equation contains a functional relationship between the displacement volume and the draft. Through this relationship, combined with the water surface distance measured by the laser rangefinder and the ship's attitude data (such as roll and pitch angles), the model can infer the ship's actual draft and calibrate any possible deviations in the visually calculated first waterline.

[0063] The coordinates of the center of buoyancy and the radius of the metacenter, defined in the equations, serve as physical priors and can be used to determine the rationality of the waterline when the ship's attitude changes. For example, when the ship rolls, the model calculates the restoring torque using the height of the metacenter and, combined with the acceleration data from the inertial measurement module, verifies whether the waterline conforms to the physical laws of the ship's equilibrium state, eliminating abnormal data caused by sensor noise or environmental interference.

[0064] The dynamic calibration model combines laser ranging data, water level gauge data, and hydrostatic curve equations to form a set of constraint equations. For example, by using the water level data at the reference point measured by the water level gauge, combined with the waterline surface area parameters corresponding to different drafts in the ship's hydrostatic curve, the change in the ship's displacement volume can be calculated. This calculation is then cross-validated with the draft derived from the laser ranging data to ensure that the multimodal data are physically consistent.

[0065] During ship navigation, the model calls the hydrostatic curve equation in real time, substitutes the three-dimensional coordinates of the draft gauge obtained by semantic segmentation and the ship attitude angles measured by inertial measurement into the equation, calculates the difference between the theoretical draft and the actual measured value, and uses algorithms such as Kalman filtering to dynamically correct the visually recognized waterline, compensating for measurement errors caused by factors such as ship swaying and load changes.

[0066] By embedding the physical prior conditions of the ship's hydrostatic curve equation, the dynamic calibration model realizes the integration of deep learning visual algorithms and ship hydrodynamic principles, enabling the water level monitoring system to not only rely on image features, but also verify the rationality of the results from the perspective of physical laws, thereby improving the accuracy and robustness of water level measurement under complex working conditions (such as ship tilting and wave interference).

[0067] For example, in step S105 above, the real-time measurement data is fused using a graph neural network for multimodal processing, including:

[0068] Laser ranging, inertial measurement, and water level gauge data are preprocessed and converted into graph-structured data. A graph neural network is constructed, using the laser ranging, inertial measurement, and water level gauge data as nodes, and the relationships between them as edges. In the graph neural network, each node absorbs information from its neighboring nodes through node feature updates and message passing mechanisms. Multiple graph convolutional layers are set to update and aggregate node features multiple times. The fused features are then converted into vectors of a unified dimension through fully connected layers, resulting in multimodal fusion features constructed from the laser ranging, inertial measurement, and water level gauge data.

[0069] Specifically, in step S105, the laser ranging data, inertial measurement data, and water level gauge data differ in physical meaning and data format. First, these data undergo preprocessing, such as normalization, to unify the data from different ranges to the same scale for subsequent processing. Then, these data are converted into graph-structured data. Using laser ranging data, inertial measurement data, and water level gauge data as nodes means that each data point is considered a node in the graph. The relationships between them are represented as edges, which can be based on physical meaning, such as the relationship between laser ranging data and ship attitude (obtained from inertial measurement data), or the relationship between water level gauge data and ship draft (related to laser ranging data). In this way, multi-source heterogeneous data is integrated into a single graph structure, providing a foundation for subsequent graph neural network processing. In the constructed graph neural network, node feature updates and message passing mechanisms are core. Each node has its own characteristics, such as the distance value of the laser ranging node and the attitude parameters of the inertial measurement node. Through the message passing mechanism, nodes can absorb information from neighboring nodes. For example, a laser ranging node can obtain ship attitude information from an inertial measurement node, because ship attitude affects the laser ranging results (e.g., the relationship between the laser-measured distance to the water surface and the actual draft changes when the ship is tilted). This information transfer allows the node to fuse relevant information from different data sources, thereby enriching its own feature representation.

[0070] Specifically, during each message transmission, nodes adjust their features based on the weights of edges with neighboring nodes (reflecting the strength of the correlation between data). For example, if laser ranging data is strongly correlated with a certain inertial measurement data (such as roll angle), the laser ranging node will absorb more information from the roll angle node during message transmission to better reflect the impact of the ship's actual state on laser ranging. Multiple graph convolutional layers are used to update and aggregate node features multiple times. Each graph convolutional layer can be viewed as a feature extraction and transformation operation on the graph structure data. In each layer, nodes are updated again based on information from neighboring nodes and their own features. As the number of layers increases, nodes can capture more complex and global information. For example, after multiple graph convolutions, a laser ranging node not only contains its own distance information and directly related ship attitude information, but may also contain indirect information related to water level gauge data (through information transmission and fusion from intermediate nodes).

[0071] This multi-layered feature update and aggregation process helps to uncover deep relationships between data, enabling models to better understand the complex interactions between multimodal data.

[0072] After processing through multiple graph convolutional layers, the node features have been sufficiently updated and fused. However, the dimensionality of these features may still be inconsistent or unsuitable for subsequent tasks. The role of the fully connected layer is to transform the fused features into a vector with a unified dimension. It integrates the features of all nodes, mapping features of different dimensions into a unified feature space through operations such as weight matrix multiplication and nonlinear activation functions. The resulting multimodal fused feature vector contains comprehensive information from laser ranging data, inertial measurement data, and water level gauge data, providing a more comprehensive reflection of the target vessel and the state of the water area it is in.

[0073] Therefore, through the graph structure representation and message passing mechanism of graph neural networks, it is possible to deeply explore the complex relationships between laser ranging data, inertial measurement data, and water level gauge data. For example, traditional methods may struggle to directly capture the joint impact of ship attitude changes on laser ranging and water level gauge measurement results, but graph neural networks can clearly reveal these relationships through information passing between nodes and feature fusion. This helps to more accurately understand the intrinsic connections between various measurement data of a ship under different states, providing richer information for subsequent analysis and decision-making.

[0074] Multimodal fusion feature vectors integrate data from multiple sensors, providing a more comprehensive and accurate description of a ship's condition compared to single data sources. For example, when laser ranging data is affected by water surface reflection, inertial measurement data and water level gauge data can be corrected and supplemented through a graph neural network fusion mechanism. This fusion of multi-source data effectively reduces the impact of single sensor failures or noise on system performance, improving the model's robustness and accuracy in various complex environments.

[0075] Ships face various complex situations during actual operation, such as different sea states (wind, waves, currents, etc.) and dynamic changes within the ship itself (changes in cargo load, adjustments to navigation attitude, etc.). Graph neural networks' multimodal fusion can better adapt to these complex scenarios. It can dynamically adjust the fusion method and weights of various data based on different ship operating states, thereby more accurately reflecting the real-time positional relationship between the ship and the water body. For example, when a ship is experiencing severe rolling, the model can more accurately calculate the ship's actual draft and waterline position through the fusion of inertial measurement data and laser ranging data, providing reliable support for safe navigation. The resulting multimodal fusion feature vector provides high-quality input for subsequent tasks (such as training and predicting dynamic calibration models).

[0076] These features can more accurately reflect the actual state of the ship, enabling the dynamic calibration model to more precisely predict the real-time positional relationship between the ship and the water area, thereby achieving more accurate calibration of the waterline. Compared with calibration using single data, calibration based on multimodal fusion features can significantly improve the accuracy and reliability of calibration, providing stronger technical support for the dynamic calibration and identification of ship draft gauges.

[0077] Furthermore, in step S105 above, a dynamic calibration model matching the target vessel is constructed based on the fusion results to predict the real-time positional relationship between the target vessel and the water area, including:

[0078] Multimodal fusion features are input into a spatiotemporal graph neural network, and the temporal dependence and spatial correlation features of the multimodal fusion features are captured through a spatiotemporal attention mechanism. A dynamic calibration model based on a Physics-aware GNN is constructed by combining the physical prior conditions of the ship's hydrostatic curve equations. A differentiable physical layer is constructed by introducing Lagrange multiplier-constrained physical laws, which include buoyancy equilibrium conditions and torque equilibrium conditions. Model parameters are optimized through an adversarial training framework, using a Physics-aware GNN as the generator and a spatiotemporal convolutional network as the discriminator to enhance the generalization ability of the dynamic calibration model to water bodies. The trained dynamic calibration model is then used to predict the real-time positional relationship.

[0079] The real-time positional relationships include at least: ship draft, ship tilt angle, roll rate, pitch rate, and center of gravity offset.

[0080] Specifically, a ship's draft refers to the depth to which it sinks in water, and is a crucial indicator for measuring a ship's load capacity and navigational safety. It directly affects the ship's buoyancy and stability, and is obtained through measurement by a laser rangefinder and calibration using other data. The ship's inclination angles, including roll and pitch angles, reflect the degree of rotation of the ship around its horizontal and vertical axes in the horizontal plane. Inertial measurement units (IMUs) collect these angular information in real time; changes in the inclination angles affect the visual representation of the draft gauge and the ship's balance. Roll and pitch angular velocities represent the speed changes of the ship's roll and pitch, also acquired by the IMU. Angular velocity information is crucial for understanding the ship's dynamic motion characteristics and helps predict future attitude changes. Center of gravity offset is the distance the ship's center of gravity deviates from its designed position due to uneven cargo distribution or other factors. Center of gravity offset significantly affects the ship's stability and is calculated by combining the ship's hydrostatic curve equations and other measurement data.

[0081] In step S105, the multimodal fusion features are input into a spatiotemporal graph neural network (STGNN), which can process data with spatiotemporal characteristics. The spatiotemporal attention mechanism plays a crucial role here, capturing the temporal dependencies of the multimodal fusion features (e.g., the changing trends of the ship's attitude at different times) and spatial correlations (e.g., the relationship between laser ranging data and the attitude of different parts of the ship). Through this mechanism, the model can better understand the changing patterns of the ship's state over time and space, thus more accurately predicting real-time positional relationships. For example, during ship rolling, the spatiotemporal attention mechanism can focus on attitude changes at different times and the spatial relationships between various sensor data, providing more comprehensive information for subsequent calibration.

[0082] By combining the physical prior conditions of the ship's hydrostatic curve equation, a dynamic calibration model based on a physics-aware graph neural network (GNN) is constructed. The ship's hydrostatic curve equation describes various physical characteristics of the ship at different drafts, such as displacement volume and center of buoyancy.

[0083] By incorporating this physical knowledge into graph neural networks, the model can reason about and predict the state of a ship based on physical principles. For example, based on the hydrostatic curve equation, the model can know information such as the ship's buoyancy and metacentric height at different drafts, thereby better understanding the ship's equilibrium state and the impact of attitude changes on draft.

[0084] Lagrange multipliers are introduced to constrain physical laws, constructing a differentiable physical layer. In the field of shipbuilding, the main considerations are the buoyancy equilibrium condition and the moment equilibrium condition. The buoyancy equilibrium condition requires that the buoyant force on the ship equals its weight, i.e., F_b = rho gV, where F_b is the buoyancy force, rho is the density of water, g is the acceleration due to gravity, and V is the displacement volume. The moment equilibrium condition ensures that the sum of the moments in all directions of the ship is zero, thus maintaining equilibrium.

[0085] By incorporating these physical laws into the model optimization process using Lagrange multipliers, the model must satisfy these physical constraints during training. For example, when calculating a ship's draft, the model considers buoyancy equilibrium conditions to ensure that the calculation results conform to physical principles.

[0086] An adversarial training framework is employed to optimize model parameters, where a physics-aware graph neural network serves as the generator, and a spatiotemporal convolutional network acts as the discriminator. The generator aims to produce ship state predictions that are as close to reality as possible, while the discriminator attempts to distinguish between the generator's predictions and actual data. Through this adversarial training approach, the generator continuously improves its predictive capabilities, enabling the dynamically calibrated model to better adapt to various complex aquatic environments and ship operating conditions. For example, during training, the generator may produce ship state predictions that do not conform to physical laws. The discriminator identifies these errors and feeds them back to the generator, prompting it to adjust its parameters and generate more realistic predictions.

[0087] Through the above steps, the dynamic calibration model can more accurately predict the real-time positional relationship between the ship and the water body. For example, after considering the ship's hydrostatic curve equations and physical constraints, the model's prediction of the ship's draft is more accurate, better reflecting the actual draft of the ship under different loads and attitudes. Simultaneously, the predictions of hull inclination angle, roll rate, pitch rate, and center of gravity shift are also more precise, providing a more reliable guarantee for the safe navigation of the ship.

[0088] The introduction of an adversarial training framework enhances the generalization ability of the dynamic calibration model across different aquatic environments and ship operating conditions. Whether in calm waters or complex sea conditions, the model can make accurate predictions based on the actual situation. For example, when facing rough seas, the model can accurately predict the ship's real-time position by learning different ship attitudes and motion patterns, unaffected by environmental noise and interference.

[0089] Because the model incorporates constraints from physical laws, its predictions are more consistent with actual physical principles. This not only improves the model's reliability but also better guides ship operation and management in practical applications. For example, when a ship is loading cargo, the model can accurately predict the ship's center of gravity shift and draft changes based on the cargo's distribution and weight, thereby helping the crew to rationally arrange cargo and ensure the ship's stability and safety.

[0090] The combination of spatiotemporal graph neural networks and dynamic calibration models enables the system to process and analyze data in real time, adapting to dynamic changes during ship operation. The model can quickly update predictions of the ship's real-time positional relationships based on the latest sensor data, providing strong support for real-time monitoring and control of the ship. For example, during navigation, the model can monitor changes in the ship's attitude in real time and adjust predictions of draft and center of gravity offset in a timely manner, allowing the crew to take timely measures to ensure safe navigation.

[0091] In summary, the dynamic calibration model constructed using the above parameters, principles, and methods can more accurately and comprehensively predict the real-time positional relationship between ships and water areas. It has high accuracy, generalization ability, and real-time performance, providing important technical support for the safe navigation and management of ships.

[0092] Optionally, in step S105, before constructing a dynamic calibration model that matches the target vessel based on the fusion results and predicting the real-time positional relationship between the target vessel and the water area, a generative adversarial network can also be constructed. The generator simulates sensor noise and occlusion scenarios, and the discriminator distinguishes between real and generated data, in order to jointly train the dynamic calibration model.

[0093] Specifically, a Generative Adversarial Network (GAN) is introduced before constructing the dynamic calibration model in step S105. The adversarial learning mechanism between the generator and discriminator enhances the model's robustness and generalization ability. The generator takes real sensor data or random noise vectors as input, simulating Gaussian noise and salt-and-pepper noise caused by water ripples and electromagnetic interference. It simulates incomplete data scenarios such as laser beam obstruction and obstructed water level gauge views through random masks or region nulling, and combines ship kinematics characteristics to ensure the generated data conforms to physical logic. The discriminator extracts the spatiotemporal features of the mixed real and generated data through convolutional neural networks or temporal models, outputting the probability of data authenticity to distinguish between the two. In adversarial training, the generator optimizes the generated data by minimizing cross-entropy loss to make it closer to the real distribution, while the discriminator simultaneously improves its discrimination accuracy, prompting the generator to improve data quality. The dynamic calibration model shares data input with the GAN, incorporating the generated noisy and incomplete data into training, learning to extract key physical features such as ship draft and inclination angle from complex data. In this process, the generator must ensure the physical correlation of multimodal data such as laser, inertial, and water level gauge data. Simultaneously, it incorporates physical prior conditions, such as the ship's hydrostatic curve, into the GAN loss function to ensure that the generated data satisfies physical laws such as buoyancy balance and torque balance. Ultimately, the dynamically calibrated model trained by GAN can effectively handle complex scenarios such as partial occlusion of laser sensors and high noise caused by severe weather, making reliable inferences using inertial measurement and water level gauge data. By expanding the distribution range of training data, the model can adapt to different sea conditions and ship operating states, reducing overfitting. Generated data based on physical constraints avoids the model learning spurious correlations, ensuring that the prediction results conform to hydrodynamic laws, providing a high-precision and highly adaptable solution for scenarios such as ship monitoring in severe weather and performance compensation of aging sensors.

[0094] Optionally, in step S105, after constructing a dynamic calibration model matching the target vessel based on the fusion results and predicting the real-time positional relationship between the target vessel and the water area, the center of gravity distribution of the target vessel can be predicted based on the real-time hull attitude data collected by the inertial measurement module; the relative positional relationship between the area where the draft gauge is located and the center of gravity distribution of the vessel can be obtained; the real-time positional relationship can be symmetrically calibrated based on the relative positional relationship; the hull roll period can be determined based on the numerical fluctuation in the relative positional relationship; and the matching real-time positional relationship value in the real-time positional relationship can be laterally calibrated based on the determination result.

[0095] Specifically, after acquiring the real-time positional relationship between the target vessel and the water area, the system uses an inertial measurement unit (IMU) and the vessel's dynamic characteristics to perform multi-dimensional calibration of the measurement results. First, the IMU collects data on the vessel's three-axis acceleration, angular velocity, and magnetic field strength. After filtering, this data is combined with the vessel's dynamic model to calculate the vessel's real-time attitude angles and acceleration, thereby retrieving the position of the vessel's center of gravity in the hull coordinate system. Then, based on the fixed installation coordinates of the draft gauge on the hull, its lateral, longitudinal, and vertical offsets from the center of gravity are calculated, clarifying the spatial relationship between the two. On this basis, utilizing the symmetrical hydrodynamic characteristics of the port and starboard sides under ideal conditions, and combining the relationship between the lateral offset of the center of gravity and the hydrostatic curve, the draft on both sides is symmetrically calibrated to eliminate measurement deviations caused by the center of gravity offset. Simultaneously, by analyzing the fluctuation patterns of the roll rate or roll angle, the ship's roll period is identified. For the periodic changes in the hull's lateral parameters during the roll process, filtering and phase correction methods are used, and the calibration coefficients are dynamically adjusted for different roll phases to suppress noise interference and ensure that the lateral position parameters conform to physical laws. This series of calibration procedures comprehensively considers the ship's motion state and physical characteristics, effectively improving the accuracy and stability of positional relationship data such as ship draft, inclination angle, and center of gravity offset, providing reliable data support for ship navigation and stability monitoring in complex sea conditions.

[0096] Step S106: Based on the real-time position relationship, dynamically calibrate the first water level line in the virtual space model to obtain the second water level line of the target ship.

[0097] For example, in step S106 above, the real-time position relationship is converted into position parameters of the virtual space model; according to the position parameters, the virtual space model is spatially transformed to simulate the actual attitude of the target ship's hull; in the transformed virtual space model, the intersection of the draft gauge area and the water surface is recalculated to obtain the calibrated water level line position; the calibrated water level line position is compared with the first water level line to calculate the position deviation; the first water level line is adjusted according to the position deviation to obtain the second water level line, which is used as the final draft gauge reading.

[0098] Specifically, step S106, which involves dynamically calibrating the first waterline in the virtual space model, is a crucial step in converting the real-time positional relationships obtained through multimodal data fusion analysis into accurate draft readings. The system first maps the real-time positional relationships, such as the ship's draft and tilt angle, predicted by fusing data from laser ranging, inertial measurement, and other methods using a graph neural network, into positional parameters for the virtual space model. These parameters encompass the ship's attitude and position information in three-dimensional space. Based on this, a spatial transformation operation is performed on the virtual space model to accurately simulate the target ship's attitude changes, such as roll and pitch, during actual navigation, ensuring the virtual model closely matches the ship's actual state. Within the adjusted virtual space model, the intersection of the draft scale area and the water surface is redefined, thus obtaining the calibrated waterline. The system compares this waterline with the first waterline initially calculated from visual images, quantifies the positional differences between the two, and then corrects the first waterline based on the deviation value, ultimately generating a second waterline reflecting the ship's actual draft. This provides a reliable waterline reading basis for ship load monitoring and navigation safety assessment.

[0099] In this embodiment, a dynamic calibration and identification method for ship draft gauges based on multimodal data fusion is used to achieve dynamic calibration of ship draft gauges, dynamically correct errors in visual recognition results, and improve the accuracy of water level lines and the efficiency of water level measurement.

[0100] Please see Figure 2 , Figure 2This application provides a ship draft gauge dynamic calibration and identification device 200 based on multimodal data fusion. The device includes: a data acquisition module for acquiring visual image data of the draft gauge scale area via a visual image module; acquiring real-time measurement data of the target ship and its surrounding water area via a laser rangefinder, an inertial measurement module, and a water level gauge; and a construction module for constructing a virtual spatial model of the draft gauge scale area based on the visual image data using a neural radiation field, and optimizing the virtual spatial model by combining pre-set physical constraints. The physical constraints include at least: ship hull operating state, ship weight distribution, etc. The system includes a cargo type and cargo capacity identification module, used for dynamic semantic segmentation of the optimized virtual space model to delineate the draft gauge area and interfering objects, and to calculate the first water level of the target vessel in real time based on the draft gauge area; a fusion module, used for multimodal fusion of the real-time measurement data through a graph neural network, constructing a dynamic calibration model matching the target vessel based on the fusion result, and predicting the real-time positional relationship between the target vessel and the water body area; the dynamic calibration model embeds the physical prior conditions of the vessel's hydrostatic curve equation; and a calibration module, used for dynamic calibration of the first water level in the virtual space model based on the real-time positional relationship to obtain the second water level of the target vessel. In some embodiments, the vessel draft gauge dynamic calibration identification device based on multimodal data fusion can be applied to terminal equipment.

[0101] It should be noted that, for the sake of convenience and brevity, the specific working process of the ship draft dynamic calibration and identification device based on multimodal data fusion described above can be referred to the corresponding process in the aforementioned embodiment of the ship draft dynamic calibration and identification method based on multimodal data fusion, and will not be repeated here.

[0102] Please see Figure 3 , Figure 3 This is a schematic block diagram of the structure of a terminal device provided in an embodiment of this application.

[0103] like Figure 3 As shown, the terminal device 300 includes a processor 301 and a memory 302, which are connected via a bus 303, such as an I2C bus. Specifically, the processor 301 provides computing and control capabilities to support the operation of the entire terminal device. The processor 301 can be a central processing unit, or it can be other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Those skilled in the art will understand that... Figure 3The structures shown are merely block diagrams of some structures related to the embodiments of this application and do not constitute a limitation on the terminal devices on which the embodiments of this application are applied. Specific servers may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. The processor is used to run a computer program stored in the memory, and when executing the computer program, implements any of the ship draft gauge dynamic calibration and identification methods based on multimodal data fusion provided in the embodiments of this application. It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the terminal device described above can be referred to the foregoing embodiments of the ship draft gauge dynamic calibration and identification method based on multimodal data fusion, and will not be repeated here.

[0104] This application also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement the steps of any of the ship draft dynamic calibration and identification methods based on multimodal data fusion provided in the specification of this application.

Claims

1. A method for dynamic calibration and identification of ship draft gauges based on multimodal data fusion, characterized in that, The method includes: The visual image module is used to collect visual image data of the water gauge scale area; Based on the visual image data, a virtual spatial model of the water gauge scale area is constructed through a neural radiation field, and the virtual spatial model is optimized in combination with pre-set physical constraints; wherein, the physical constraints include at least: ship hull operating status, ship weight distribution, cargo type, and cargo volume. Dynamic semantic segmentation is performed on the optimized virtual space model to divide the water gauge scale area and interference objects, and the first water level line of the target ship is calculated in real time based on the water gauge scale area. Real-time measurement data of the target vessel and the water area it is in are obtained through laser rangefinders, inertial measurement modules, and water level gauges. The real-time measurement data is fused into multiple modes using a graph neural network, including: preprocessing laser ranging data, inertial measurement data, and water level gauge data to convert them into graph-structured data; constructing a graph neural network with laser ranging data, inertial measurement data, and water level gauge data as nodes, and the relationships between them as edges; in the graph neural network, each node absorbs information from its neighboring nodes through node feature updates and message passing mechanisms; setting multiple graph convolutional layers to update and aggregate node features multiple times; and converting the fused features into vectors of a unified dimension through a fully connected layer to obtain the multimodal fused features constructed from laser ranging data, inertial measurement data, and water level gauge data. Based on the fusion results, a dynamic calibration model matching the target ship is constructed to predict the real-time positional relationship between the target ship and the water area. The dynamic calibration model is embedded with the physical prior conditions of the ship's hydrostatic curve equation. The real-time positional relationship includes at least: ship draft, hull inclination angle, roll rate, pitch rate, and center of gravity offset. Based on the real-time positional relationship, the first water level line in the virtual space model is dynamically calibrated to obtain the second water level line of the target ship.

2. The method according to claim 1, characterized in that, The process of constructing a virtual spatial model of the water level scale region based on the visual image data using a neural radiation field, and optimizing the virtual spatial model in conjunction with pre-set physical constraints, includes: Acquire multi-view images from the visual image data, and extract image feature points using a feature extraction network; Sparse point cloud reconstruction based on feature points generates an initial 3D point cloud model of the water gauge scale region. The initial 3D point cloud model is input into the neural radiation field model, and the neural radiation field is converted into a renderable virtual space model through volume rendering technology. Based on the pre-set physical constraints corresponding to the ship's operating state, ship's weight distribution, cargo type, and cargo volume, a comprehensive constraint loss function is constructed to optimize the parameters of the neural radiation field model. The optimized parameters are then re-input into the neural radiation field model to generate a virtual space model that includes the water gauge scale region.

3. The method according to claim 2, characterized in that, The process of dynamically segmenting the optimized virtual space model to delineate the draft gauge area and interfering objects, and calculating the first water level of the target vessel in real time based on the draft gauge area, includes: The virtual space model containing the water gauge scale area is divided into three-dimensional grid units to obtain a semantic segmentation network containing multiple three-dimensional grid units; Based on the spatiotemporal attention mechanism, the three-dimensional grid cells in the semantic segmentation network are classified to identify the water gauge scale area, rust, water ripples, and hull. Edge detection is performed on the segmented water gauge scale area to extract the scale line contour; the height difference between adjacent scale lines is calculated according to the preset scale line spacing and scale standard, and the scale line contour is optimized using the height difference to obtain the three-dimensional position coordinates of the water gauge scale; By combining the transformation relationship between the image coordinate system and the world coordinate system, the three-dimensional position coordinates are converted into actual physical coordinates, and the first water level line is calculated in real time based on the physical coordinates of the water gauge scale.

4. The method according to claim 1, characterized in that, The process of acquiring real-time measurement data of the target vessel and its surrounding water area using a laser ranging sensor, an inertial measurement module, and a water level gauge includes: Initialize the laser rangefinder sensor, setting the measurement frequency and measurement range; During the operation of the target vessel, the laser rangefinder emits a laser beam into the water area, receives the reflected laser signal, and calculates the distance from the sensor to the water surface based on the flight time. The inertial measurement module collects the three-axis acceleration, angular velocity and magnetic field strength data of the target ship in real time, and removes noise through a filtering algorithm to calculate the ship's attitude angle. Install the water level gauge at a static reference point to measure the vertical distance between the reference point and the water surface in real time; The laser ranging data, inertial measurement data, and water level gauge data are timestamped to form real-time measurement data of the target vessel and the water area it is in.

5. The method according to claim 4, characterized in that, The dynamic calibration model, constructed based on the fusion results and matched to the target vessel, predicts the real-time positional relationship between the target vessel and the water area, including: Multimodal fusion features are input into a spatiotemporal graph neural network, and the temporal dependence and spatial correlation features of the multimodal fusion features are captured through a spatiotemporal attention mechanism; By combining the physical prior conditions of the ship hydrostatic curve equation, a dynamic calibration model based on the Physics-aware GNN is constructed. A differentiable physical layer is constructed by introducing the Lagrange multiplier constraint law; the Lagrange multiplier constraint law includes buoyancy equilibrium condition and torque equilibrium condition; The model parameters are optimized by using an adversarial training framework, and a physical perception graph neural network is used as the generator and a spatiotemporal convolutional network as the discriminator to enhance the generalization ability of the dynamic calibration model to water bodies. The trained dynamic calibration model is used to predict the real-time positional relationship.

6. The method according to claim 5, characterized in that, Before constructing a dynamic calibration model matching the target vessel based on the fusion results to predict the real-time positional relationship between the target vessel and the water area, the following steps are also included: Generative adversarial networks are constructed to simulate sensor noise and occlusion scenarios through a generator, and to distinguish between real and generated data through a discriminator, in order to jointly train a dynamic calibration model.

7. The method according to claim 5, characterized in that, After constructing a dynamic calibration model matching the target vessel based on the fusion results and predicting the real-time positional relationship between the target vessel and the water area, the method further includes: Based on the real-time hull attitude data collected by the inertial measurement module, the center of gravity distribution of the target ship is predicted. Obtain the relative positional relationship between the area where the draft gauge is located and the distribution of the ship's center of gravity; Based on the relative positional relationship, the real-time positional relationship is calibrated for symmetry. Based on the numerical fluctuations in the relative positional relationship, the ship's roll period is determined, and the real-time positional relationship value matched in the real-time positional relationship is laterally calibrated based on the determination result.

8. The method according to claim 1, characterized in that, The step of dynamically calibrating the first waterline in the virtual space model based on the real-time positional relationship to obtain the second waterline of the target vessel includes: The real-time location relationship is converted into location parameters of a virtual space model; Based on the position parameters, the virtual space model is spatially transformed to simulate the actual attitude of the target ship's hull. In the transformed virtual space model, the intersection of the water gauge scale area and the water surface is recalculated to obtain the calibrated water level line position. The calibrated water level line position is compared with the first water level line, and the position deviation is calculated; The first water level line is adjusted according to the positional deviation to obtain the second water level line, which serves as the final water gauge reading.

9. A ship draft dynamic calibration and identification system based on multimodal data fusion, characterized in that, The system includes the following modules: The acquisition module is used to acquire visual image data of the water level gauge area through the visual image module; and to obtain real-time measurement data of the target vessel and the water area it is in through the laser rangefinder, inertial measurement module and water level gauge. The construction module is used to construct a virtual spatial model of the water gauge scale area based on the visual image data through a neural radiation field, and optimize the virtual spatial model in combination with pre-set physical constraints; wherein, the physical constraints include at least: ship hull operating state, ship weight distribution, cargo type, and cargo load. The identification module is used to perform dynamic semantic segmentation on the optimized virtual space model, divide the water gauge scale area and interference objects, and calculate the first water level line of the target ship in real time based on the water gauge scale area. The fusion module is used to perform multimodal fusion of the real-time measurement data through a graph neural network, and to construct a dynamic calibration model that matches the target ship based on the fusion result, predicting the real-time positional relationship between the target ship and the water area. The dynamic calibration model is embedded with the physical prior conditions of the ship's hydrostatic curve equation, and the real-time positional relationship includes at least: ship draft, hull inclination angle, roll rate, pitch rate, and center of gravity offset. The fusion module, when fusing the real-time measurement data through a graph neural network for multimodal fusion, specifically preprocesses the laser ranging data, inertial measurement data, and water level gauge data to convert them into graph-structured data; constructs a graph neural network, using the laser ranging data, inertial measurement data, and water level gauge data as nodes, and the relationships between them as edges; in the graph neural network, through node feature updates and message passing mechanisms, each node absorbs information from its neighboring nodes; sets multiple graph convolutional layers to update and aggregate node features multiple times; and converts the fused features into vectors of a unified dimension through a fully connected layer to obtain the multimodal fused features constructed from the laser ranging data, inertial measurement data, and water level gauge data. The calibration module is used to dynamically calibrate the first water level line in the virtual space model based on the real-time position relationship, so as to obtain the second water level line of the target ship.

Images