Ship water gauge dynamic calibration identification system and method based on multi-modal data fusion

Abstract

The invention belongs to the field of image processing, and provides a ship water gauge dynamic calibration recognition system and method based on multi-modal data fusion. The method comprises the following steps: collecting visual image data of a water gauge scale area; a virtual space model of the water gauge scale area is constructed based on the visual image data, and the virtual space model is optimized in combination with preset physical constraints; performing dynamic semantic segmentation on the virtual space model, dividing a water gauge scale area and an interferent, and calculating a first water line of the target ship in real time based on the water gauge scale area; acquiring real-time measurement data of the target ship and the water body area where the target ship is located; performing multi-modal fusion on the real-time measurement data through a graph neural network, constructing a dynamic calibration model based on a fusion result, and predicting a real-time position relationship between the target ship and the water body region; and dynamically calibrating the first water line based on the real-time position relationship to obtain a second water line. Errors of visual identification results are dynamically corrected, and measurement accuracy and measurement efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a ship water gauge dynamic calibration and identification system and method based on multimodal data fusion. Background Art

[0002] A water gauge is a scale device that directly observes changes in water levels in water bodies. It reflects the height difference of the water surface relative to a reference point through scale markings and is widely used in water conservancy projects, hydrological monitoring, waterway management and other fields.

[0003] Long-term loading or improper loading can cause distortion in a ship's hull (such as a mid-section sag or partial bulge), causing the scale to deviate from the actual draft. For example, deformation in one ship's midsection resulted in a draft difference of 12cm between port and starboard. Furthermore, rust or wear on the hull's steel plates can blur or offset the scale markings, affecting accurate readings.

[0004] Therefore, in response to the above technical problems, it is urgent to design a new technical solution to solve at least one of the above technical problems. Summary of the Invention

[0005] The main purpose of the embodiments of the present application is to provide a ship water gauge dynamic calibration and identification system and method based on multimodal data fusion, aiming to solve the technical problem in related technologies of reduced water gauge accuracy caused by hull deformation or corrosion.

[0006] In a first aspect, an embodiment of the present application provides a method for dynamic calibration and identification of a ship draft gauge based on multimodal data fusion, comprising:

[0007] Collect visual image data of the water gauge scale area through the visual image module;

[0008] Based on the visual image data, a virtual space model of the water gauge scale area is constructed through the neural radiation field, and the virtual space model is optimized in combination with pre-set physical constraints; wherein the physical constraints include at least: the operating state of the hull, the gravity distribution of the hull, the cargo type, and the cargo capacity;

[0009] Dynamic semantic segmentation is performed on the optimized virtual space model to divide the water gauge scale area and interference objects, and the first water level line of the target ship is calculated in real time based on the water gauge scale area;

[0010] Obtain real-time measurement data of the target ship and the water area it is in through laser ranging sensors, inertial measurement modules and water level gauges;

[0011] The real-time measurement data is subjected to multimodal fusion through a graph neural network, and a dynamic calibration model matching the target ship is constructed based on the fusion results to predict the real-time position relationship between the target ship and the water area; the dynamic calibration model is embedded with the physical prior conditions of the ship's hydrostatic curve equation;

[0012] Based on the real-time position relationship, the first water level in the virtual space model is dynamically calibrated to obtain a second water level of the target ship.

[0013] In a second aspect, an embodiment of the present application provides a ship water gauge dynamic calibration and identification system based on multimodal data fusion, comprising:

[0014] The acquisition module is used to collect visual image data of the water gauge scale area through the visual image module; and obtain real-time measurement data of the target ship and the water area it is in through the laser ranging sensor, inertial measurement module and water level meter;

[0015] a construction module for constructing a virtual space model of the draft scale area based on the visual image data using a neural radiation field, and optimizing the virtual space model in combination with pre-set physical constraints; wherein the physical constraints include at least: hull operating state, hull gravity distribution, cargo type, and cargo capacity;

[0016] The recognition module is used to perform dynamic semantic segmentation on the optimized virtual space model, divide the water gauge scale area and interference objects, and calculate the first water level line of the target ship in real time based on the water gauge scale area;

[0017] A fusion module is configured to perform multimodal fusion of the real-time measurement data through a graph neural network, construct a dynamic calibration model that matches the target ship based on the fusion results, and predict the real-time positional relationship between the target ship and the water area; the dynamic calibration model is embedded with the physical prior conditions of the ship's hydrostatic curve equation;

[0018] The calibration module is used to dynamically calibrate the first water level line in the virtual space model based on the real-time position relationship to obtain the second water level line of the target ship.

[0019] In a third aspect, an embodiment of the present application further provides a terminal device, comprising a processor and a memory for storing a computer program; the processor is configured to execute the computer program and implement the ship water gauge dynamic calibration and identification method based on multimodal data fusion as described in the first aspect or any embodiment of the present application when executing the computer program.

[0020] The embodiments of the present application provide a ship water gauge dynamic calibration and identification system and method based on multimodal data fusion. In the method, visual image data of a draft scale area is collected through a visual image module; based on the visual image data, a virtual space model of the draft scale area is constructed through a neural radiation field, and the virtual space model is optimized in combination with pre-set physical constraints; wherein the physical constraints include at least: hull operating status, hull gravity distribution, cargo type, and cargo capacity; dynamic semantic segmentation is performed on the optimized virtual space model to divide the draft scale area and interference objects, and the first water level line of the target ship is calculated in real time based on the draft scale area; real-time measurement data of the target ship and the water area in which it is located is obtained through a laser ranging sensor, an inertial measurement module, and a water level meter; the real-time measurement data is multimodally fused through a graph neural network, and a dynamic calibration model matching the target ship is constructed based on the fusion result to predict the real-time position relationship between the target ship and the water area; the dynamic calibration model is embedded with the physical prior conditions of the ship's hydrostatic curve equation; based on the real-time position relationship, the first water level line in the virtual space model is dynamically calibrated to obtain the second water level line of the target ship.

[0021] This method can effectively make up for the defect of traditional pure visual recognition that is easily misjudged due to environmental interference through the deep integration of physical data such as laser ranging and inertial measurement with visual information. At the same time, it uses a dynamic calibration model embedded in the ship's hydrostatic curve equation to achieve real-time correction of the water level line in scenarios such as cargo changes. In terms of improving accuracy, a high-precision virtual space model is constructed with the help of neural radiation fields and physical constraints, and water gauge scale information is accurately extracted through dynamic semantic segmentation. Relying on graph neural networks to realize real-time automatic processing of multimodal data, combined with intelligent prediction and dynamic calibration mechanisms, the efficiency of water level measurement is significantly improved. In addition, the use of generative adversarial networks to enhance model training enables the system to have good robustness and environmental adaptability under different ship types, sea conditions and lighting conditions, breaking through traditional technical bottlenecks in all aspects and providing a more reliable and efficient solution for ship water level measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a flow chart of a method for dynamic calibration and identification of a ship draft gauge based on multimodal data fusion provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of the module structure of a ship water gauge dynamic calibration and identification system based on multimodal data fusion provided in an embodiment of the present application;

[0024] Figure 3 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] Embodiments of the present application provide a system and method for dynamic calibration and identification of ship draft gauges based on multimodal data fusion. This method can be applied to a terminal device, which can be a mobile terminal such as a mobile phone, virtual reality device, tablet computer, laptop computer, desktop computer, wearable device, or other electronic device. The terminal device can be a cloud server connected to a ship management system or a server cluster. This connection can be implemented via hardware circuitry or a communication module.

[0026] The following is a detailed description of some embodiments of the present application in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Figure 1 , Figure 1 A flow chart of a method for dynamic calibration and identification of a ship water gauge based on multimodal data fusion provided in an embodiment of the present application.

[0027] like Figure 1 As shown, the ship water gauge dynamic calibration and identification method based on multimodal data fusion includes the following steps:

[0028] Step S101: Collect visual image data of the water gauge scale area through the visual image module.

[0029] In the embodiment of the present application, visual image data refers to image information collected by the visual image module and used for water gauge scale identification. It is collected in the form of a video stream or a single-frame image by a high-definition camera (such as an industrial-grade area array camera, a high-resolution surveillance camera). The camera can be installed in a fixed position on the deck of the ship to shoot the water gauge from a top-down or side-view angle; it can also be deployed at a shore-based monitoring point to obtain ship water gauge images at a long distance to meet monitoring needs in different scenarios. The image must completely cover the ship's water gauge scale area, including the digital scale of the water gauge, scale lines, and a certain range of the hull surface around the water gauge. At the same time, the waterline where the water gauge meets the water surface is captured to provide a key basis for subsequent water level calculations.

[0030] To ensure accurate recognition, the visual image data must have a high resolution to ensure that the details of the water gauge scale are clearly discernible. Regarding frame rates, for real-time monitoring of dynamic scenes, a frame rate of 25 frames per second or higher is recommended to avoid motion blur. Furthermore, excellent color reproduction and a wide dynamic range are required to accommodate varying lighting conditions (such as direct sunlight and nighttime lighting).

[0031] The original images after collection usually need to be preprocessed, including but not limited to denoising (removing noise and snow interference in the image), geometric correction (correcting image distortion caused by shooting angle), brightness and contrast adjustment, etc., to improve image quality and lay the foundation for subsequent steps such as virtual space model construction based on neural radiation field and dynamic semantic segmentation.

[0032] Under different lighting conditions, faced with problems such as water gauge reflections and shadow interference caused by direct strong light, or image blurring caused by insufficient light at night, the data collected by the visual image module may experience reduced clarity and color distortion. However, the system uses a multimodal data fusion mechanism to introduce physical draft depth data obtained by the laser ranging sensor, which can directly avoid the interference of light on visual recognition and ensure the accuracy of water level measurement. At the same time, the virtual space model constructed by the neural radiation field combined with physical constraints is not affected by changes in lighting, can continuously provide stable water gauge scale spatial information, and cooperate with dynamic semantic segmentation technology to accurately extract the water gauge area. In addition, the generative adversarial network pre-simulates the impact of different lighting scenarios on the data, and the trained dynamic calibration model has strong robustness, which enables the technology to maintain stable and reliable measurement performance under lighting conditions such as day and night alternation and changes in sunny and cloudy weather.

[0033] Step S102: Based on the visual image data, a virtual space model of the water ruler scale area is constructed through the neural radiation field, and the virtual space model is optimized in combination with pre-set physical constraints.

[0034] In the embodiment of the present application, the physical constraints include at least: the hull operating state, the hull gravity distribution, the cargo type, and the cargo capacity.

[0035] Specifically, in the embodiments of the present application, physical constraints such as the hull's operating status, hull's gravity distribution, cargo type, and cargo capacity provide accurate basis for the dynamic calibration and identification of the ship's water level gauge from different dimensions.

[0036] Hull operating status: This includes information on the ship's roll, pitch, and bow pitch, as well as motion parameters such as navigation speed and acceleration. The hull's attitude data is collected in real time through the inertial measurement unit (IMU), providing the system with real-time attitude information of the hull in three-dimensional space. When the ship tilts, the visual presentation of the draft scale will be distorted due to perspective. Traditional pure visual recognition is prone to errors. However, combined with the constraints of the hull's operating status, the draft scale attitude in the virtual space model can be adjusted synchronously to ensure that the direction of the draft scale is consistent with the actual hull attitude, eliminating waterline positioning deviations caused by hull shaking.

[0037] Hull gravity distribution: Closely linked to the ship's structural design and cargo loading position, it determines the ship's force equilibrium. Different gravity distributions can cause varying degrees of sinking and tilting. Based on the ship's design drawings and real-time cargo loading, the system calculates the hull's gravity distribution, converts it into constraints, and integrates them into the virtual space model. This ensures that the draft gauge position in the model matches the actual force state of the hull, avoiding draft gauge reading errors caused by uneven gravity distribution and ensuring that the measurement results reflect the ship's true mechanical equilibrium.

[0038] Cargo type: Different types of cargo (such as bulk cargo, containers, and liquid cargo) have different stacking characteristics and center of gravity distribution patterns. Bulk cargo may shift due to sloshing during navigation, changing the ship's center of gravity; liquid cargo has a free surface effect, affecting ship stability. Based on the characteristics of cargo type, the system optimizes the virtual space model and adjusts the calculation method of the draft scale and water level line to adapt the model to the impact of different cargo types on the ship's draft depth and posture, thereby improving recognition accuracy and reliability.

[0039] Cargo capacity directly determines a ship's draft and is a key factor affecting draft gauge readings. As cargo is loaded and unloaded, the ship's cargo capacity and draft change accordingly. The system acquires cargo capacity data in real time and, combined with the ship's hydrostatic curve equation, dynamically adjusts the draft gauge scale position and draft in the virtual space model. This enables real-time updates of the waterline during the loading process, ensuring that draft gauge measurements accurately and promptly reflect changes in the ship's cargo status.

[0040] Exemplarily, in step S102, based on the visual image data, a virtual space model of the water ruler scale area is constructed through the neural radiation field, and the virtual space model is optimized in combination with pre-set physical constraints, including:

[0041] Acquire multi-view images from the visual image data, extract image feature points through a feature extraction network; perform sparse point cloud reconstruction based on the feature points to generate an initial three-dimensional point cloud model of the draft scale area; input the initial three-dimensional point cloud model into a neural radiation field model, and convert the neural radiation field into a renderable virtual space model through volume rendering technology; construct a comprehensive constraint loss function based on the pre-set physical constraints corresponding to the hull operating state, hull gravity distribution, cargo type, and cargo capacity, and optimize the parameters of the neural radiation field model; re-input the optimized parameters into the neural radiation field model to generate a virtual space model including the draft scale area.

[0042] Specifically, in step S102, the process of constructing and optimizing the virtual space model of the water gauge scale area based on visual image data is a key link in achieving high-precision modeling through the integration of multiple technologies.

[0043] The multi-view images collected by the visual image module cover information about the water gauge scale area at different angles. Through feature extraction networks (such as classic algorithms such as SIFT, SURF, ORB, or models such as SuperPoint and LoFTR based on deep learning), unique and stable feature points can be extracted from the image. These feature points contain key information such as the inflection points of the water gauge scale lines and the edges of the digital contours, providing the basis for subsequent three-dimensional reconstruction. For example, the SuperPoint model uses convolutional neural networks to automatically learn feature points and their descriptors in images, which has higher accuracy and robustness than traditional methods in complex lighting and texture environments.

[0044] Based on the extracted feature points, a feature matching algorithm (such as brute force matching or FLANN matching) is used to find corresponding feature point pairs in the multi-view images. Triangulation is then used to calculate the coordinates of these feature points in 3D space, thereby generating an initial 3D point cloud model of the water gauge scale area. This process converts 2D image information into 3D spatial information, initially restoring the geometric shape of the water gauge. However, the point cloud is relatively sparse, containing only the spatial locations of the feature points.

[0045] The initial 3D point cloud model is input into the Neural Radiance Field (NeRF) model. NeRF uses a multi-layer perceptron (MLP) to learn the volume density and radiance (color) information of each point in the scene, constructing a continuous 3D scene representation. During training, by inputting the direction and position of light rays from different perspectives, NeRF predicts the color and opacity at the intersection of the light rays and the scene. This information is then accumulated using volume rendering technology to generate a renderable virtual space model. Volume rendering technology can simulate the propagation and absorption of light in a scene, ultimately outputting realistic 3D model images that accurately restore the texture and geometric details of the water level scale.

[0046] For example, when constructing a comprehensive constraint loss function, physical constraints such as the hull's operating state, hull gravity distribution, cargo type, and cargo capacity can be combined. Specifically, the hull roll angle, pitch angle, bow angle, and other attitude data collected in real time by the inertial measurement unit (IMU) are used, and the difference between the hull attitude parameters in the model and the IMU measurement values ​​is used as a constraint term to ensure that the direction of the water gauge scale in the model is consistent with the actual hull attitude, avoiding visual deformation of the scale caused by hull tilt. The hull gravity distribution is calculated based on the ship design drawings and the real-time cargo loading situation. A constraint term reflecting the balance of gravity distribution is introduced into the comprehensive constraint loss function to ensure that the water gauge position in the model matches the actual force state of the hull, avoiding water gauge reading errors caused by uneven gravity distribution. Considering the different effects of different cargo types (such as bulk cargo, containerized cargo, and liquid cargo) on the center of gravity and stability of the ship, constraints related to the cargo type are added to the comprehensive constraint loss function, the model is optimized according to the characteristics of different cargo types, and the calculation method of the water gauge scale and water level line is adjusted. Combined with the ship's hydrostatic curve equation, the relationship between cargo capacity and draft is incorporated into the comprehensive constraint loss function. The draft scale position and draft in the virtual space model are dynamically corrected based on the difference between the draft in the comprehensive constraint loss function and the theoretical draft corresponding to the actual cargo capacity.

[0047] Therefore, the aforementioned physical constraints are converted into corresponding loss terms and combined with the visual reconstruction loss of the neural radiation field model (i.e., the difference between the rendered image and the original visual image) to form a comprehensive constraint loss function. The parameters of the neural radiation field model are iteratively optimized through a backpropagation algorithm to minimize the comprehensive constraint loss function, resulting in a virtual space model that conforms to physical laws and closely matches the actual scene.

[0048] Furthermore, in an optional example, the aforementioned physical constraints can be converted into corresponding loss terms through a comprehensive constraint loss function. This is then combined with the visual reconstruction loss of the neural radiation field model (i.e., the difference between the rendered image and the original visual image) to form a comprehensive constraint loss function. The parameters of the neural radiation field model are iteratively optimized through a backpropagation algorithm to minimize the comprehensive constraint loss function, thereby obtaining a virtual space model that conforms to physical laws and closely matches the actual scene.

[0049] Finally, the optimized neural radiation field model parameters were re-entered into the model, and the updated volume density and radiosity information was used to generate the final virtual space model through volume rendering technology. This model not only accurately reproduces the geometric and texture details of the water gauge scale area, but also fully considers the influence of physical factors such as the hull's operating state, gravity distribution, cargo type and cargo volume, providing an accurate and reliable foundation for subsequent dynamic semantic segmentation and waterline calculation.

[0050] Step S103: Dynamic semantic segmentation is performed on the optimized virtual space model to divide the water gauge scale area and the interference object, and the first water level line of the target ship is calculated in real time based on the water gauge scale area.

[0051] Exemplarily, in the above step S103, the virtual space model containing the water gauge scale area is divided into three-dimensional grid units to obtain a semantic segmentation network containing multiple three-dimensional grid units; based on the spatiotemporal attention mechanism, the three-dimensional grid units in the semantic segmentation network are classified to identify the water gauge scale area, rust, water ripples, and hull; edge detection is performed on the segmented water gauge scale area, and the scale line contour is extracted; according to the preset scale line spacing and scale standard, the height difference between adjacent scale lines is calculated, and the height difference is used to optimize the scale line contour to obtain the three-dimensional position coordinates of the water gauge scale; combined with the conversion relationship between the image coordinate system and the world coordinate system, the three-dimensional position coordinates are converted into actual physical coordinates, and the first water level line is calculated in real time based on the physical coordinates of the water gauge scale.

[0052] Specifically, in step S103, dynamic semantic segmentation and first waterline calculation rely on three-dimensional modeling, deep learning and geometric calculation to achieve accurate recognition.

[0053] First, the optimized virtual space model is divided into a uniform 1cm×1cm×1cm grid of cubes. The continuous model is discretized using the Marching Cubes algorithm or volume rendering sampling to reduce computational complexity. Each grid cell carries geometric coordinates, visual color, and volume density values, as well as associated physical parameters such as the ship's posture and gravity distribution. Secondly, a 3D semantic segmentation network, such as U-Net3D or PointNet++, is constructed and embedded with a spatiotemporal attention module. Spatially, a self-attention mechanism is used to capture the continuity of scale lines. Temporally, an RNN or 3D convolution is used to distinguish static scale lines from dynamic interference, enabling accurate classification of water gauge scale areas, rust marks, and water ripples. Next, edge detection is performed on the segmented scale areas using the 3D Canny operator (Canny3D) or the 3D Sobel operator (Sobel3D). The spatial edge features of the scale lines are extracted through 3D gradient calculations to generate an initial contour point cloud. To address the potential noise in the point cloud, the team combined the standard size characteristics of the main scale lines (10cm spacing) and auxiliary scale lines (1cm spacing) to first classify spatially adjacent points into potential scale line groups using a clustering algorithm (such as DBSCAN). Each point cloud group was then fitted with a line or curve using the least squares method. Finally, noise points that deviated from the fitted model were removed, and the precise three-dimensional coordinates of the scale lines were optimized. Canny3D, a three-dimensional extension of the two-dimensional Canny algorithm, performs edge detection through a series of steps: first, Gaussian filtering is performed on the volume data in three-dimensional space to smooth it and reduce noise interference; then, a three-dimensional difference operator is used to calculate the gradient magnitude and direction of each voxel; then, non-maximum suppression is performed on the gradient direction, retaining local maxima to refine the edge; finally, double-threshold segmentation is used to distinguish strong from weak edges, retaining only weak edges connected to strong edges, ultimately obtaining continuous three-dimensional edges. Sobel3D calculates the gradient of volume data based on a three-dimensional differential convolution kernel. By applying the Sobel operator in the x, y, and z directions, it obtains the gradient amplitude and direction of each voxel in three-dimensional space. Compared with Canny3D, Sobel3D is simpler to implement and faster to calculate, but it is more sensitive to noise, and the edge extraction results may contain more redundant information. Finally, based on the camera calibration parameters, the world coordinate system and the image coordinate system are converted, the intersection of the waterline and the scale line is located in the image, and then reversely projected to the world coordinate system. Combined with laser ranging data or the ship's hydrostatic curve, the vertical coordinates of the water level are calibrated to determine the physical position of the first water level.

[0054] Therefore, by relying on three-dimensional semantic segmentation to enhance the ability to resist rust and water ripple interference, using spatiotemporal attention to track the hull shaking, and deeply integrating physical constraints to ensure that the results meet the design specifications, a solid foundation is laid for subsequent multimodal calibration.

[0055] Step S104: Acquire real-time measurement data of the target ship and the water area in which it is located through a laser ranging sensor, an inertial measurement module, and a water level meter.

[0056] For example, in step S104, real-time measurement data of the target vessel and the water area in which it is located is obtained by using a laser ranging sensor, an inertial measurement module, and a water level meter, including:

[0057] Initialize the laser ranging sensor and set the measurement frequency and range. During the operation of the target ship, the laser ranging sensor emits a laser beam into the water area, receives the reflected laser signal, and calculates the distance from the sensor to the water surface based on the flight time. The inertial measurement module collects the target ship's three-axis acceleration, angular velocity, and magnetic field intensity data in real time, removes noise through a filtering algorithm, and calculates the ship's attitude angle. A water level meter is installed at a static reference point to measure the vertical distance between the reference point and the water surface in real time. The laser ranging data, inertial measurement data, and water level meter data are timestamped to form real-time measurement data of the target ship and the water area in which it is located.

[0058] Specifically, in step S104, the acquisition of real-time measurement data is the basis of multimodal fusion, and its core lies in the collaborative collection of key physical information through the laser ranging sensor, inertial measurement module and water level gauge. During the startup phase, the laser ranging sensor needs to be initialized and the appropriate measurement frequency (such as 10 times per second) and measurement range (covering the maximum draft of the ship) are set according to the actual application scenario to ensure that it can work stably. During the operation of the ship, the laser ranging sensor continuously emits a laser beam to the water area. By capturing the flight time from the laser emission to the reflection back to the sensor, the vertical distance from the sensor to the water surface is accurately calculated. This data directly reflects the current draft of the ship. The inertial measurement module collects the ship's three-axis acceleration, angular velocity and magnetic field intensity data in real time. Since the raw data is often mixed with environmental noise and equipment errors, it needs to be processed by algorithms such as Kalman filtering or complementary filtering. After effectively removing noise interference, the real-time roll angle, pitch angle and bow angle of the ship are further calculated to accurately obtain the ship's attitude information. If a water level gauge is used, it must be installed at a static reference point unaffected by the vessel's motion. A built-in level sensor measures the vertical distance between the reference point and the water surface in real time, providing a reference for water level calculations. Finally, to ensure temporal consistency across multiple data sources, millisecond-accurate timestamps must be added to the laser ranging data, inertial measurement data, and water level gauge data. These data must then be integrated into a complete, real-time measurement dataset of the target vessel and the water area it is located in. This provides reliable physical data support for subsequent multimodal data fusion and dynamic calibration.

[0059] Step S105: The real-time measurement data is subjected to multimodal fusion through a graph neural network, and a dynamic calibration model matching the target ship is constructed based on the fusion results to predict the real-time position relationship between the target ship and the water area.

[0060] In the embodiment of the present application, the dynamic calibration model is embedded with physical prior conditions of the ship's hydrostatic curve equation.

[0061] Physical prior conditions are designed to combine the ship's fluid dynamics characteristics with real-time measurement data to improve the accuracy and reliability of water level monitoring. Specifically, the ship's hydrostatic curve equation is a mathematical model that describes the relationship between parameters such as the ship's displacement volume, center of buoyancy, metacentric radius, and draft in still water. This equation is typically pre-constructed based on ship design drawings and fluid dynamics calculations, reflecting the ship's physical properties and hydrodynamic laws.

[0062] The dynamic calibration model embeds these physical prerequisites into the model architecture as equations. The hydrostatic curve equation includes a functional relationship between displacement volume and draft depth. Using this relationship, combined with the water surface distance measured by the laser ranging sensor and vessel attitude data (such as roll and pitch angles), the model can infer the actual draft of the vessel and calibrate any deviations from the visually calculated first waterline.

[0063] The center of buoyancy coordinates and metacentric radius defined in the equation serve as physical priors and can be used to determine the rationality of the waterline when the ship's attitude changes. For example, when the ship rolls, the model calculates the righting moment from the metacentric height. Combined with the acceleration data from the inertial measurement module, it verifies that the waterline conforms to the physical laws of the ship's equilibrium state, eliminating any anomalies caused by sensor noise or environmental interference.

[0064] The dynamic calibration model combines laser ranging data, water level gauge data, and hydrostatic curve equations to form a set of constrained equations. For example, the water level data at the reference point measured by the water level gauge, combined with the waterplane area parameters corresponding to different drafts in the ship's hydrostatic curve, can be used to calculate the change in the ship's displacement volume. This is then cross-validated with the draft derived from the laser ranging data to ensure physical consistency of the multimodal data.

[0065] During the ship's navigation, the model calls the hydrostatic curve equation in real time, substitutes the three-dimensional coordinates of the water gauge scale obtained by semantic segmentation and the ship's attitude angle obtained by inertial measurement into the equation, calculates the difference between the theoretical draft and the actual measured value, and dynamically corrects the visually recognized water level line through algorithms such as Kalman filtering to compensate for measurement errors caused by factors such as ship shaking and load changes.

[0066] By embedding the physical prior conditions of the ship's hydrostatic curve equation, the dynamic calibration model realizes the integration of deep learning visual algorithms and ship fluid mechanics principles, so that the water level monitoring system not only relies on image features, but also can verify the rationality of the results from the level of physical laws, thereby improving the water level measurement accuracy and robustness under complex working conditions (such as ship tilt and wave interference).

[0067] For example, in step S105, multimodal fusion of the real-time measurement data through a graph neural network may include:

[0068] The laser ranging data, inertial measurement data and water level meter data are preprocessed and converted into graph structure data; a graph neural network is constructed with the laser ranging data, inertial measurement data and water level meter data as nodes and the correlation between the laser ranging data, inertial measurement data and water level meter data as edges; in the graph neural network, each node absorbs the information of adjacent nodes through node feature update and message passing mechanism; a multi-layer graph convolution layer is set to update and aggregate the node features multiple times; the fused features are converted into vectors of unified dimension through a fully connected layer to obtain multimodal fusion features constructed by laser ranging data, inertial measurement data and water level meter data.

[0069] Specifically, in step S105, the laser ranging data, inertial measurement data, and water level gauge data differ in physical meaning and data format. These data are first preprocessed, such as normalization, to unify data from different ranges to the same scale for subsequent processing. Then, these data are converted into graph-structured data. Using laser ranging data, inertial measurement data, and water level gauge data as nodes means that each data point is considered a node in the graph. The relationships between them are edges, and these relationships can be based on physical meaning, such as the relationship between laser ranging data and ship attitude (obtained from inertial measurement data), or the relationship between water level gauge data and ship draft (related to laser ranging data). In this way, multi-source heterogeneous data are integrated into a graph structure, providing a foundation for subsequent graph neural network processing. In the constructed graph neural network, node feature updates and message passing mechanisms are the core. Each node has its own characteristics, such as the distance value of the laser ranging node and the attitude parameters of the inertial measurement node. Through the message passing mechanism, nodes can absorb information from neighboring nodes. For example, a laser ranging node can obtain ship attitude information from an inertial measurement node, as the ship's attitude affects the results of laser ranging (for example, the relationship between the laser-measured surface distance and the actual draft changes when the hull tilts). This information transfer enables nodes to fuse relevant information from different data sources, thereby enriching their own feature representations.

[0070] Specifically, during each message transmission, a node adjusts its own features based on the weights of the edges between adjacent nodes (reflecting the strength of the association between the data). For example, if the laser ranging data is strongly correlated with a specific inertial measurement data point (such as roll angle), the laser ranging node will incorporate more information from the roll angle node during message transmission to better reflect the impact of the ship's actual state on the laser ranging. Multiple layers of graph convolution are used to update and aggregate node features multiple times. Each layer of graph convolution can be viewed as a feature extraction and transformation operation on the graph-structured data. In each layer, the node is again updated based on the information of adjacent nodes and its own features. As the number of layers increases, the node is able to capture more complex and global information. For example, after multiple layers of graph convolution, the laser ranging node not only contains its own distance information and directly related ship posture information, but may also contain indirect information related to the water level gauge data (through information transmission and fusion between intermediate nodes).

[0071] This multi-layer feature updating and aggregation process helps to explore deep relationships between data, enabling the model to better understand the complex interactions between multimodal data.

[0072] After processing through multiple layers of graph convolutional layers, node features are fully updated and fused. However, the feature dimensions may still be inconsistent or unsuitable for subsequent tasks. The fully connected layer converts these fused features into vectors of uniform dimension. It integrates the features of all nodes and, through operations such as weight matrix multiplication and nonlinear activation functions, maps features of different dimensions into a unified feature space. The resulting multimodal fused feature vector incorporates comprehensive information from laser ranging data, inertial measurement data, and water level gauge data, providing a more comprehensive picture of the target vessel and the water area in which it resides.

[0073] Therefore, through the graph structure representation and message passing mechanism of graph neural networks, it is possible to deeply explore the complex relationships between laser ranging data, inertial measurement data, and water level gauge data. For example, traditional methods may not be able to directly capture the joint impact of changes in ship posture on laser ranging and water level gauge measurements, but graph neural networks can clearly reveal these relationships through information transmission and feature fusion between nodes. This helps to more accurately understand the inherent connections between various measurement data under different ship states, providing richer information for subsequent analysis and decision-making.

[0074] Multimodal fusion feature vectors combine data from multiple sensors, providing a more comprehensive and accurate description of the ship's status than single data sources. For example, when laser ranging data is deviated by interference from water reflections, inertial measurement data and water level gauge data can be corrected and supplemented through the fusion mechanism of graph neural networks. This multi-source data fusion effectively reduces the impact of single sensor failures or noise on system performance, improving the model's robustness and accuracy in various complex environments.

[0075] In actual operation, ships face various complex situations, such as different sea conditions (wind, waves, currents, etc.) and the dynamic changes of the ship itself (changes in cargo capacity, adjustments to navigation attitude, etc.). The multimodal fusion of graph neural networks can better adapt to these complex scenarios. It can dynamically adjust the fusion method and weight of various data according to the different operating states of the ship, so as to more accurately reflect the real-time position relationship between the ship and the water area. For example, when the ship shakes violently, the model can more accurately calculate the actual draft and water level position of the ship by fusing inertial measurement data and laser ranging data, providing reliable support for the safe navigation of the ship. The obtained multimodal fusion feature vector provides high-quality input for subsequent tasks (such as training and prediction of dynamic calibration models).

[0076] These features can more accurately reflect the actual state of the ship, enabling the dynamic calibration model to more precisely predict the real-time positional relationship between the ship and the water area, thereby achieving more accurate calibration of the water level. Compared with calibration using a single data set, calibration based on multimodal fusion features can significantly improve calibration accuracy and reliability, providing more powerful technical support for dynamic calibration and identification of ship water gauges.

[0077] Furthermore, in the above step S105, a dynamic calibration model matching the target ship is constructed based on the fusion result to predict the real-time position relationship between the target ship and the water area, including:

[0078] The multimodal fusion features are input into the spatiotemporal graph neural network, and the temporal dependency and spatial correlation characteristics of the multimodal fusion features are captured through the spatiotemporal attention mechanism; combined with the physical prior conditions of the ship's hydrostatic curve equation, a dynamic calibration model based on the physics-aware graph neural network Physics-aware GNN is constructed; the Lagrange multiplier constrained physical law is introduced to construct a differentiable physical layer; the Lagrange multiplier constrained physical law includes buoyancy balance conditions and torque balance conditions; the model parameters are optimized through an adversarial training framework, and the physics-aware graph neural network is used as the generator and the spatiotemporal convolutional network is used as the discriminator to enhance the generalization ability of the dynamic calibration model for water bodies; the trained dynamic calibration model is used to predict the real-time position relationship.

[0079] The real-time position relationship includes at least: ship draft, hull tilt angle, roll angular velocity, pitch angular velocity, and center of gravity offset.

[0080] Specifically, a ship's draft refers to the depth to which it sinks in water and is a key indicator of its load capacity and navigation safety. It directly affects the ship's buoyancy and stability and is measured using laser ranging sensors and calibrated with other data. The hull's inclination angles, including roll and pitch angles, reflect the degree of rotation of the ship about its transverse and longitudinal axes in the horizontal plane. The inertial measurement module collects these angles in real time. Changes in the hull's inclination angles affect the visual presentation of the draft scale and the ship's equilibrium. The roll and pitch angular velocities, which represent the changes in the hull's rolling and pitching speeds, are also acquired by the inertial measurement module. Angular velocity information is crucial for understanding the ship's dynamic motion characteristics and helps predict future changes in the ship's attitude. The center of gravity offset refers to the distance the ship's center of gravity deviates from its designed position due to uneven cargo distribution or other factors. This center of gravity offset can significantly affect the ship's stability and is calculated by combining the ship's hydrostatic curve equation with other measured data.

[0081] In step S105, the multimodal fusion features are input into a spatiotemporal graph neural network (STGNN), which is capable of processing data with spatiotemporal characteristics. The spatiotemporal attention mechanism plays a key role in this. It can capture the dependencies of multimodal fusion features in time series (such as the changing trend of the ship's posture at different times) and spatial correlation features (such as the relationship between laser ranging data and the posture of each part of the ship). Through this mechanism, the model can better understand the changing laws of the ship's state over time and space, thereby more accurately predicting the real-time position relationship. For example, during the ship's shaking process, the spatiotemporal attention mechanism can pay attention to the posture changes at different times and the spatial connection between the sensor data, providing more comprehensive information for subsequent calibration.

[0082] A dynamic calibration model based on a physics-aware GNN was constructed by combining the physical priors of the ship's hydrostatic curve equation. The ship's hydrostatic curve equation describes various physical characteristics of the ship at different drafts, such as displacement volume and center of buoyancy.

[0083] Integrating this physical knowledge into graph neural networks enables the model to reason and predict the ship's state based on physical principles. For example, based on the hydrostatic curve equation, the model can determine information such as the ship's buoyancy and metacentric height at different draft depths, thereby better understanding the ship's equilibrium state and the impact of attitude changes on draft depth.

[0084] Lagrange multipliers are introduced to constrain the laws of physics and construct a differentiable physics layer. In the field of ships, the main considerations are buoyancy and torque balance conditions. The buoyancy balance condition requires that the buoyancy force on a ship equals its weight, i.e., F_b = rho gV, where F_b is the buoyancy force, rho is the density of water, g is the acceleration due to gravity, and V is the displacement volume. The torque balance condition ensures that the sum of the moments on the ship in all directions is zero, maintaining equilibrium.

[0085] These physical laws are incorporated into the model's optimization process through Lagrange multipliers, ensuring that the model meets these physical constraints during training. For example, when calculating a ship's draft, the model considers buoyancy equilibrium conditions to ensure that the calculation results conform to physical principles.

[0086] An adversarial training framework is used to optimize model parameters, with a physics-aware graph neural network serving as the generator and a spatiotemporal convolutional network as the discriminator. The generator aims to produce ship state predictions that are as close to reality as possible, while the discriminator attempts to distinguish between the generator's predictions and real data. Through this adversarial training approach, the generator continuously improves its prediction capabilities, enabling the dynamic calibration model to better adapt to various complex water environments and ship operating conditions. For example, during training, the generator may generate some ship state predictions that do not conform to physical laws. The discriminator identifies these errors and provides feedback to the generator, prompting it to adjust its parameters to produce more realistic predictions.

[0087] Through these steps, the dynamic calibration model can more accurately predict the real-time positional relationship between the ship and the water area. For example, by considering the ship's hydrostatic curve equation and physical constraints, the model more accurately predicts the ship's draft, better reflecting the ship's actual draft under different loads and attitudes. Furthermore, predictions of the ship's inclination angle, roll angular velocity, pitch angular velocity, and center of gravity offset are also more precise, providing a more reliable guarantee for the ship's safe navigation.

[0088] The introduction of an adversarial training framework enables the dynamic calibration model to generalize more effectively to diverse water environments and ship operating conditions. Whether in calm waters or complex sea conditions, the model can accurately predict the actual situation. For example, in rough seas, the model can accurately predict the real-time position of ships by learning different ship postures and motion patterns, without being affected by environmental noise and interference.

[0089] Because the model incorporates the constraints of physical laws, its predictions are more consistent with actual physical principles. This not only enhances the model's credibility but also provides better guidance for ship operations and management in practical applications. For example, when loading cargo, the model can accurately predict the ship's center of gravity offset and draft changes based on the distribution and weight of the cargo, helping the crew to arrange the cargo appropriately and ensure the ship's stability and safety.

[0090] The combination of a spatiotemporal graph neural network and a dynamic calibration model enables the system to process and analyze data in real time, adapting to dynamic changes during a vessel's operation. The model can rapidly update its predictions of the vessel's real-time positional relationships based on the latest sensor data, providing strong support for real-time monitoring and control of the vessel. For example, during navigation, the model can monitor changes in the vessel's attitude in real time and promptly adjust its predictions of draft and center of gravity offset, enabling the crew to take timely action to ensure safe navigation.

[0091] In summary, the dynamic calibration model constructed by the above parameters, principles and methods can more accurately and comprehensively predict the real-time position relationship between ships and water areas. It has high accuracy, generalization ability and real-time performance, and provides important technical support for the safe navigation and management of ships.

[0092] Optionally, in step S105, a dynamic calibration model matching the target ship is constructed based on the fusion result. Before predicting the real-time position relationship between the target ship and the water area, a generative adversarial network can be constructed to simulate sensor noise and occlusion scenes through the generator and distinguish between real and generated data through the discriminator to jointly train the dynamic calibration model.

[0093] Specifically, before the construction of the dynamic calibration model in step S105, a generative adversarial network (GAN) is introduced to enhance the model's robustness and generalization capabilities through the adversarial learning mechanism between the generator and the discriminator. The generator takes real sensor data or random noise vectors as input, simulates Gaussian noise and salt-and-pepper noise caused by water fluctuations and electromagnetic interference, and simulates incomplete data scenarios such as laser beam occlusion and obstructed water level gauge field of view through random masks or regional zeroing. It also combines the ship's kinematic characteristics to ensure that the generated data conforms to physical logic; the discriminator extracts the spatiotemporal features of the mixed real and generated data through convolutional neural networks or time series models, and outputs the probability of data authenticity to distinguish between the two. During adversarial training, the generator optimizes the generated data by minimizing the cross-entropy loss to make it closer to the real distribution, while the discriminator simultaneously improves the discrimination accuracy, prompting the generator to improve data quality. The dynamic calibration model shares data input with the GAN, incorporates the generated noisy and incomplete data into training, and learns to extract key physical features such as ship draft and tilt angle from complex data. During this process, the generator needs to ensure the physical correlation of multimodal data such as laser, inertial, and water level gauges, while incorporating physical prior conditions such as the ship's hydrostatic curve into the GAN loss function to ensure that the generated data satisfies physical laws such as buoyancy balance and torque balance. Ultimately, the dynamic calibration model trained by GAN can effectively cope with complex scenarios such as partial occlusion of the laser sensor and high noise caused by severe weather, and use inertial measurement and water level gauge data for reliable inference. By expanding the distribution range of training data, the model can adapt to different sea conditions and ship operating states, reducing overfitting. Generating data based on physical constraints avoids false correlations in model learning, ensuring that the prediction results conform to the laws of hydrodynamics, and providing high-precision and highly adaptable solutions for scenarios such as ship monitoring in severe weather and performance compensation of old sensors.

[0094] Optionally, in step S105, a dynamic calibration model matching the target ship is constructed based on the fusion result, and after predicting the real-time position relationship between the target ship and the water area, the hull center of gravity distribution of the target ship can be predicted based on the real-time hull posture data collected by the inertial measurement module; the relative position relationship between the position area of ​​the water gauge and the hull center of gravity distribution is obtained; based on the relative position relationship, the real-time position relationship is symmetrically calibrated; based on the numerical fluctuations in the relative position relationship, the hull roll period is judged, and based on the judgment result, the matching real-time position relationship value in the real-time position relationship is laterally calibrated.

[0095] Specifically, after obtaining the real-time positional relationship between the target vessel and the water area, the system utilizes the inertial measurement module and the ship's dynamic characteristics to perform a multi-dimensional calibration of the measurement results. First, the IMU collects the ship's triaxial acceleration, angular velocity, and magnetic field strength data. After filtering and combining it with the ship's dynamics model, the system infers the ship's real-time attitude angle and acceleration, thereby inverting the position of the ship's center of gravity in the ship's hull coordinate system. Subsequently, based on the fixed coordinates of the water level gauge on the hull, the system calculates its lateral, longitudinal, and vertical offsets from the center of gravity to clarify the spatial relationship between the two. Based on this, the system utilizes the ideal hydrodynamic symmetry between the port and starboard sides of the ship and combines the relationship between the lateral center of gravity offset and the hydrostatic curve to perform a symmetrical calibration of the port and starboard drafts, eliminating measurement errors caused by the center of gravity offset. Furthermore, by analyzing the fluctuation pattern of the roll angular velocity or roll angle, the system identifies the ship's roll period. Filtering and phase correction techniques are used to address the periodic changes in the ship's transverse parameters during roll. Dynamically adjusting the calibration coefficients based on different roll phases suppresses noise interference and ensures that the transverse position parameters conform to physical laws. This series of calibration processes comprehensively considers the ship's motion state and physical characteristics, effectively improving the accuracy and stability of positional relationship data such as the ship's draft, inclination angle, and center of gravity offset, providing reliable data support for ship navigation and stability monitoring in complex sea conditions.

[0096] Step S106: Based on the real-time position relationship, dynamically calibrate the first water level in the virtual space model to obtain a second water level of the target ship.

[0097] Exemplarily, in the above-mentioned step S106, the real-time position relationship is converted into position parameters of the virtual space model; according to the position parameters, the virtual space model is spatially transformed to simulate the actual posture of the hull of the target ship; in the transformed virtual space model, the intersection of the water gauge scale area and the water surface is recalculated to obtain the calibrated water level line position; the calibrated water level line position is compared with the first water level line to calculate the position deviation; the first water level line is adjusted according to the position deviation to obtain the second water level line as the final water gauge reading.

[0098] Specifically, in step S106, dynamic calibration of the first waterline in the virtual space model is a key step in converting the real-time positional relationships derived from multimodal data fusion analysis into accurate water gauge readings. The system first maps the real-time positional relationships, such as the ship's draft and hull inclination angle, predicted through graph neural network fusion of laser ranging and inertial measurement data, into positional parameters for the virtual space model. These parameters encompass the ship's attitude and position in three-dimensional space. Based on this, spatial transformation operations are performed on the virtual space model to accurately simulate the target ship's actual rolling and pitching posture changes during navigation, ensuring that the virtual model closely matches the ship's actual state. Within the adjusted virtual space model, the intersection of the water gauge scale area and the water surface is redefined to obtain a calibrated waterline. The system compares this waterline with the first waterline initially calculated from visual images, quantifies the positional difference between the two, and then corrects the first waterline based on the deviation. Ultimately, a second waterline reflecting the ship's actual draft is generated, providing a reliable basis for water level readings for ship load monitoring and navigation safety assessment.

[0099] In the embodiment of the present application, a dynamic calibration and recognition method of a ship draft gauge based on multimodal data fusion is used to achieve dynamic calibration of the ship draft gauge, dynamically correct errors in visual recognition results, and improve the accuracy of the water level line and the efficiency of water level measurement.

[0100] See also Figure 2 , Figure 2The embodiment of the present application provides a ship draft gauge dynamic calibration and identification device 200 based on multimodal data fusion, which includes: an acquisition module for acquiring visual image data of the draft gauge scale area through a visual image module; obtaining real-time measurement data of the target ship and the water area in which it is located through a laser ranging sensor, an inertial measurement module and a water level meter; a construction module for constructing a virtual space model of the draft gauge scale area through a neural radiation field based on the visual image data, and optimizing the virtual space model in combination with pre-set physical constraints; wherein the physical constraints include at least: the hull operation state, the hull gravity distribution, Cargo type and cargo volume; an identification module for dynamically semantically segmenting the optimized virtual space model, dividing the draft gauge scale area and interference objects, and calculating the first water level of the target ship in real time based on the draft gauge scale area; a fusion module for multimodally fusing the real-time measurement data through a graph neural network, constructing a dynamic calibration model that matches the target ship based on the fusion results, and predicting the real-time position relationship between the target ship and the water area; the dynamic calibration model is embedded with the physical prior conditions of the ship's hydrostatic curve equation; a calibration module for dynamically calibrating the first water level in the virtual space model based on the real-time position relationship to obtain the second water level of the target ship. In some embodiments, a ship draft gauge dynamic calibration and identification device based on multimodal data fusion can be applied to a terminal device.

[0101] It should be noted that, for the convenience and conciseness of description, the specific working process of the ship draft gauge dynamic calibration and identification device based on multimodal data fusion described above can refer to the corresponding process in the aforementioned embodiment of the ship draft gauge dynamic calibration and identification method based on multimodal data fusion, and will not be repeated here.

[0102] See also Figure 3 , Figure 3 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application.

[0103] like Figure 3 As shown, the terminal device 300 includes a processor 301 and a memory 302, which are connected via a bus 303, such as an I2C bus. Specifically, the processor 301 is used to provide computing and control capabilities to support the operation of the entire terminal device. The processor 301 can be a central processing unit, or it can be other general-purpose processors, digital signal processors, application-specific integrated circuits, field programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It will be understood by those skilled in the art that Figure 3The structure shown in is only a block diagram of a part of the structure related to the embodiment of the present application, and does not constitute a limitation on the terminal device to which the embodiment of the present application is applied. The specific server may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components. Among them, the processor is used to run the computer program stored in the memory, and implement any one of the ship water gauge dynamic calibration and identification methods based on multimodal data fusion provided in the embodiment of the present application when executing the computer program. It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the terminal device described above can refer to the aforementioned embodiment of the ship water gauge dynamic calibration and identification method based on multimodal data fusion, and will not be repeated here.

[0104] An embodiment of the present application also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any step of the ship water gauge dynamic calibration and identification method based on multimodal data fusion as provided in the description of the embodiment of the present application.

Claims

1. A ship water gauge dynamic calibration and identification method based on multimodal data fusion, characterized in that: The method comprises: Collect visual image data of the water gauge scale area through the visual image module; Based on the visual image data, a virtual space model of the water gauge scale area is constructed through the neural radiation field, and the virtual space model is optimized in combination with pre-set physical constraints; wherein the physical constraints include at least: the operating state of the hull, the gravity distribution of the hull, the cargo type, and the cargo capacity; Dynamic semantic segmentation is performed on the optimized virtual space model to divide the water gauge scale area and interference objects, and the first water level line of the target ship is calculated in real time based on the water gauge scale area; Obtain real-time measurement data of the target ship and the water area it is in through laser ranging sensors, inertial measurement modules and water level gauges; The real-time measurement data is subjected to multimodal fusion through a graph neural network, and a dynamic calibration model matching the target ship is constructed based on the fusion results to predict the real-time position relationship between the target ship and the water area; the dynamic calibration model is embedded with the physical prior conditions of the ship's hydrostatic curve equation; Based on the real-time position relationship, the first water level in the virtual space model is dynamically calibrated to obtain a second water level of the target ship.

2. The method according to claim 1, characterized in that The method of constructing a virtual space model of the water ruler scale area based on the visual image data through the neural radiation field and optimizing the virtual space model in combination with pre-set physical constraints includes: Acquire multi-view images from the visual image data, and extract image feature points through a feature extraction network; Reconstruct sparse point cloud based on feature points to generate an initial 3D point cloud model of the water gauge scale area; The initial 3D point cloud model is input into the neural radiation field model, and the neural radiation field is converted into a renderable virtual space model through volume rendering technology; Based on the pre-set physical constraints corresponding to the hull's operating state, hull's gravity distribution, cargo type, and cargo capacity, a comprehensive constraint loss function is constructed to optimize the parameters of the neural radiation field model. The optimized parameters are re-input into the neural radiation field model to generate a virtual space model containing the water ruler scale area.

3. The method according to claim 2, characterized in that The method of performing dynamic semantic segmentation on the optimized virtual space model to divide the water gauge scale area and the interference object, and calculating the first water level line of the target ship in real time based on the water gauge scale area, includes: The virtual space model containing the water gauge scale area is divided into three-dimensional grid units to obtain a semantic segmentation network containing multiple three-dimensional grid units; Based on the spatiotemporal attention mechanism, the 3D grid cells in the semantic segmentation network are classified to identify the water gauge scale area, rust, water ripples, and hull; Perform edge detection on the segmented water gauge scale area and extract the scale line outline; calculate the height difference between adjacent scale lines based on the preset scale line spacing and scale standard, and use the height difference to optimize the scale line outline to obtain the three-dimensional position coordinates of the water gauge scale; Combined with the conversion relationship between the image coordinate system and the world coordinate system, the three-dimensional position coordinates are converted into actual physical coordinates, and the first water level line is calculated in real time based on the physical coordinates of the water gauge scale.

4. The method according to claim 1, wherein The method of obtaining real-time measurement data of the target ship and the water area in which it is located by using a laser ranging sensor, an inertial measurement module, and a water level meter includes: Initialize the laser ranging sensor and set the measurement frequency and measurement range; When the target ship is in motion, the laser ranging sensor emits a laser beam into the water area, receives the reflected laser signal, and calculates the distance from the sensor to the water surface based on the flight time; The inertial measurement module collects the target ship's three-axis acceleration, angular velocity, and magnetic field strength data in real time, removes noise through a filtering algorithm, and calculates the ship's attitude angle; Install the water level meter at a static reference point to measure the vertical distance between the reference point and the water surface in real time; The laser ranging data, inertial measurement data and water level meter data are time-stamped to form real-time measurement data of the target ship and the water area in which it is located.

5. The method according to claim 4, characterized in that The performing multimodal fusion of the real-time measurement data through a graph neural network includes: Preprocess the laser ranging data, inertial measurement data and water level meter data and convert them into graph structure data; Construct a graph neural network with laser ranging data, inertial measurement data, and water level meter data as nodes and the relationships between them as edges. In graph neural networks, each node absorbs information from adjacent nodes through node feature updates and message passing mechanisms; Set up multi-layer graph convolution layers to update and aggregate node features multiple times; The fused features are converted into vectors of uniform dimension through the fully connected layer to obtain multimodal fusion features constructed by laser ranging data, inertial measurement data and water level meter data.

6. The method according to claim 5, characterized in that The method of constructing a dynamic calibration model matching the target ship based on the fusion results and predicting the real-time position relationship between the target ship and the water area includes: The multimodal fusion features are input into the spatiotemporal graph neural network, and the temporal dependency and spatial correlation features of the multimodal fusion features are captured through the spatiotemporal attention mechanism; Combined with the physical prior conditions of the ship's hydrostatic curve equation, a dynamic calibration model based on the Physics-aware GNN is constructed; Introducing the Lagrange multiplier constraint physical law to construct a differentiable physical layer; the Lagrange multiplier constraint physical law includes buoyancy balance conditions and torque balance conditions; The model parameters are optimized through an adversarial training framework, using a physical-aware graph neural network as the generator and a spatiotemporal convolutional network as the discriminator to enhance the generalization ability of the dynamic calibration model for water bodies. Using the trained dynamic calibration model to predict the real-time position relationship; The real-time position relationship includes at least: ship draft, hull tilt angle, roll angular velocity, pitch angular velocity, and center of gravity offset.

7. The method according to claim 6, characterized in that Before constructing a dynamic calibration model matching the target ship based on the fusion results and predicting the real-time position relationship between the target ship and the water area, the method further includes: A generative adversarial network is constructed to simulate sensor noise and occlusion scenarios through the generator, and distinguish between real and generated data through the discriminator, which are used to jointly train a dynamic calibration model.

8. The method according to claim 6, characterized in that After constructing a dynamic calibration model matching the target ship based on the fusion results and predicting the real-time position relationship between the target ship and the water area, the method further includes: Predict the target ship's hull center of gravity distribution based on the real-time hull attitude data collected by the inertial measurement module; Obtain the relative position relationship between the water gauge location area and the center of gravity distribution of the ship; Based on the relative position relationship, performing symmetry calibration on the real-time position relationship; Based on the numerical fluctuations in the relative position relationship, the hull roll period is judged, and based on the judgment result, the matching real-time position relationship values ​​in the real-time position relationship are calibrated laterally.

9. The method according to claim 1, characterized in that The dynamically calibrating the first water level in the virtual space model based on the real-time position relationship to obtain the second water level of the target ship includes: Converting the real-time position relationship into position parameters of a virtual space model; According to the position parameters, the virtual space model is spatially transformed to simulate the actual posture of the target ship; In the transformed virtual space model, the intersection point between the water gauge scale area and the water surface is recalculated to obtain the calibrated water level position; Comparing the calibrated water level position with the first water level position to calculate the position deviation; The first water level line is adjusted according to the position deviation to obtain the second water level line as the final water gauge reading.

10. A ship water gauge dynamic calibration and identification system based on multimodal data fusion, characterized in that: The system includes the following modules: The acquisition module is used to collect visual image data of the water gauge scale area through the visual image module; and obtain real-time measurement data of the target ship and the water area it is in through the laser ranging sensor, inertial measurement module and water level meter; a construction module for constructing a virtual space model of the draft scale area based on the visual image data using a neural radiation field, and optimizing the virtual space model in combination with pre-set physical constraints; wherein the physical constraints include at least: hull operating state, hull gravity distribution, cargo type, and cargo capacity; The recognition module is used to perform dynamic semantic segmentation on the optimized virtual space model, divide the water gauge scale area and interference objects, and calculate the first water level line of the target ship in real time based on the water gauge scale area; A fusion module is configured to perform multimodal fusion of the real-time measurement data through a graph neural network, construct a dynamic calibration model that matches the target ship based on the fusion results, and predict the real-time positional relationship between the target ship and the water area; the dynamic calibration model is embedded with the physical prior conditions of the ship's hydrostatic curve equation; The calibration module is used to dynamically calibrate the first water level line in the virtual space model based on the real-time position relationship to obtain the second water level line of the target ship.

Images