Digital twinning risk early warning system based on channel multi-source data fusion

Abstract

The invention relates to the technical field of channel safety management, and more specifically relates to a channel multi-source data fusion digital twinborn risk early warning system comprising a multi-source data acquisition module used for acquiring channel specific data types, comprising AIS data, radar data, video monitoring data, hydro meteorological data, channel surveying and mapping data, ship report data, shore-based sensor data and unmanned aerial vehicle patrol data, and comprehensive data support is provided for channel management through space-time alignment, dynamic weight adjustment and conflict resolution; a high-fidelity digital twin is constructed, fluid dynamics and a ship behavior model are fused, and a real channel scene is restored; intelligent risk early warning and traceability are realized by using deep learning, and decision scientificity is improved; efficient virtual-real interaction is realized through multi-terminal pushing and sand table deduction, and a management closed loop is formed; a cloud-edge-end framework and the like are adopted to optimize system performance, and real-time performance and safety in a large-scale scene are guaranteed.

Description

Technical Field

[0001] This invention relates to the field of waterway safety management technology, and more specifically to a digital twin risk early warning system that integrates multi-source waterway data. Background Technology

[0002] With the rapid development of the global shipping industry, waterway traffic is becoming increasingly busy, and the trend towards larger and faster ships is evident. The waterway environment is complex and ever-changing, posing numerous challenges to ship navigation safety. Waterway safety management involves multi-dimensional information such as ship dynamic monitoring, hydro-meteorological monitoring, and waterway facility status perception. Traditional management methods are no longer sufficient to meet the demands of modern shipping safety. Currently, the main technical bottlenecks are as follows:

[0003] (1) Insufficient ability to fuse multi-source heterogeneous data

[0004] Existing waterway monitoring systems typically employ multiple data sources, including AIS, radar, video surveillance, and hydrological sensors. However, due to differences in data formats, acquisition frequencies, and spatiotemporal references (e.g., AIS data drift, radar multipath interference, and video data affected by visibility), effective fusion of multi-source data is difficult. For example:

[0005] Although the automatic identification and monitoring method and supervision system for ships at sea that integrates multiple data sources, disclosed in CN103971543B, has achieved the fusion of static data from RFID, video and laser scanning, it has not solved the problem of real-time coupling between dynamic environmental data (such as water flow and wind waves) and ship motion.

[0006] The port area multi-source sensing system proposed in the intelligent monitoring system for port area and waterway based on multi-source information fusion, with publication number CN104064009A, relies on GPRS transmission, has a high data packet loss rate under severe weather conditions, and lacks an intelligent mechanism for resolving conflicting data.

[0007] (2) The risk warning model has a low level of intelligence.

[0008] Traditional early warning systems are mostly based on fixed thresholds or simple rules (such as safe distance thresholds and speed limits), which cannot adapt to the dynamic complexity of waterway scenarios. For example:

[0009] A discrete waterway traffic information system with publication number CN113129646A calculates safety through waterway curvature and density parameters, but does not consider the impact of sudden weather events (such as fog or strong winds) on ship maneuverability.

[0010] (3) The application of digital twin technology is superficial.

[0011] Existing digital twin systems mostly focus on three-dimensional visualization and lack accurate simulation of the dynamic characteristics of the physical world. For example:

[0012] A method and system for predicting equipment failure based on digital twins, with publication number CN118350228B, achieves equipment-level failure prediction, but does not extend to the macroscopic simulation of waterway traffic systems.

[0013] Although the urban safety risk management system based on digital twin technology, with publication number CN110929923A, integrates satellite and drone data, the dynamic response of the twin deviates significantly from that of the real ship, making it unable to support emergency simulations.

[0014] Based on this, the present invention proposes a digital twin risk early warning system for waterway multi-source data fusion to solve the problems existing in the above-mentioned background technology. Summary of the Invention

[0015] The purpose of this invention is to provide a digital twin risk early warning system for waterway multi-source data fusion, which solves the technical problems mentioned in the background art.

[0016] The objective of this invention can be achieved through the following technical solutions:

[0017] A digital twin risk early warning system for waterway multi-source data fusion includes:

[0018] The multi-source data acquisition module is used to collect waterway-specific data types, including AIS data, radar data, video surveillance data, hydrological and meteorological data, waterway mapping data, ship report data, shore-based sensor data, and UAV patrol data.

[0019] The multi-source data fusion processing module includes a spatiotemporal benchmark unification engine, a channel scenario-driven fusion strategy unit, and a conflict resolution and data completion unit.

[0020] The unified spatiotemporal reference engine establishes a unified spatiotemporal coordinate system through the following formula, thereby achieving spatiotemporal alignment of multi-source heterogeneous data:

[0021]

[0022] Among them, P adjusted For the adjusted ship position, P original For the original ship position,

[0023] t0 is the start time, t is the current time, and V current (t) is the water flow velocity vector that varies with time; high-precision time synchronization technology is used to ensure time consistency of all data sources, and spatial data is converted to a unified coordinate system based on waterway GIS data;

[0024] The fusion strategy unit driven by the waterway scenario dynamically adjusts the fusion weights w of each data source according to different regions, weather conditions, and traffic density using the following formula. i :

[0025] w i =f(S,C i )

[0026] Where S represents the current set of waterway scenario parameters, including region type, weather conditions, and traffic density, and C i Let f represent the set of feature parameters of the i-th data source, and f be the fusion weight calculation function;

[0027] The conflict resolution and data completion unit utilizes knowledge graphs or deep learning models to identify and resolve sensor conflicts, and employs deep learning models such as Transformer to predictively complete missing data. The Transformer model based on the attention mechanism is calculated as follows:

[0028]

[0029] Where Q is the query vector, K is the key vector, V is the value vector, and d is the value vector. k The dimension of the key vector;

[0030] The digital twin construction module includes a multi-scale fluid dynamics coupling model unit, an agent-driven ship behavior modeling unit, and an environmental element dynamic integration unit.

[0031] A multi-scale fluid dynamics coupled model unit, integrating macroscopic hydrological models and microscopic CFD models, performs microscopic CFD calculations based on the Navier-Stokes equations at bridge piers, wharves, and narrow waterways. The formulas are as follows:

[0032]

[0033] Where ρ is the fluid density, v is the velocity vector, t is time, p is pressure, μ is dynamic viscosity, and f is external force; the macroscopic hydrological model and the microscopic CFD model are coupled through data transfer and boundary conditions to achieve high-precision water flow simulation.

[0034] The agent-driven ship behavior modeling unit constructs an autonomous agent for each ship based on a physical model and intelligent rules. The ship maneuvering physical model adopts the MMG (Mathematical Model Group) model, and its equations of motion are as follows:

[0035]

[0036] Where m is the ship's mass, u and v are the ship's longitudinal and lateral velocities respectively, r is the ship's bow angular velocity, and X... G For the longitudinal coordinate of the ship's center of gravity, I z Let X be the moment of inertia of the ship about the z-axis, and let X, Y, and N be the longitudinal force, lateral force, and bow moment acting on the ship, respectively.

[0037] The dynamic integration unit for environmental elements integrates real-time environmental data on meteorological, visibility, and water depth changes, and dynamically influences the ship motion model through the following formula:

[0038]

[0039] in, This is the actual acceleration vector of the ship. The acceleration vector generated by the ship's own power. This represents the acceleration vector caused by environmental factors.

[0040] The risk warning and source tracing analysis module includes a multimodal fusion risk feature extraction unit, a dynamic risk assessment and graded early warning unit, and a risk source tracing and root cause analysis unit.

[0041] The multimodal fusion risk feature extraction unit utilizes a spatiotemporal graph neural network (ST-GNN) for risk feature extraction, where nodes v in the graph structure... i The feature update formula is as follows:

[0042]

[0043] in, For node v i The feature vector at layer l+1, where σ is the activation function and N(i) is the node v. i The set of neighboring nodes, W l Let b be the weight matrix of the l-th layer. l Let l be the bias vector of the l-th layer;

[0044] The dynamic risk assessment and hierarchical early warning unit combines a deep learning prediction model and a rule / knowledge graph-based inference engine for dynamic risk assessment. The deep learning prediction model uses an LSTM network, and its cell state update formula is as follows:

[0045]

[0046] Among them, f t For the output of the forget gate, i t For input gate output, For candidate cell states, C t In cellular state, o t For output gate output, ht To be in a hidden state, W f W i W c W o Let b be the weight matrix. f b i b C b o It is the bias vector;

[0047] The risk tracing and root cause analysis unit uses explainable AI technology or reasoning paths to trace and explain the causes of risk formation.

[0048] The virtual-real interaction module includes a multi-terminal intelligent adaptation and push unit, a digital twin sand table simulation unit, and an early warning-handling-feedback closed-loop unit.

[0049] The multi-terminal intelligent adaptation and push unit intelligently pushes early warning information to different terminals based on risk level, vessel type, location, and crew role, and provides differentiated operation suggestions;

[0050] The digital twin sand table simulation unit simulates issuing instructions, adjusting environmental parameters, and simulating ship responses on the twin when a high-risk event is warned, in order to evaluate the effectiveness of different emergency plans and potential chain risks.

[0051] The early warning-response-feedback closed-loop unit feeds back actual response instructions and results to the digital twin, optimizing the early warning model and rule base;

[0052] The system architecture optimization module includes a cloud-edge-device collaborative computing architecture unit, a lightweight twin rendering and interaction unit, and a secure and reliable mechanism unit.

[0053] The cloud-edge-device collaborative computing architecture unit is designed with a layered processing architecture. The sensor / ship end performs preliminary data filtering and lightweight processing, the edge computing node is responsible for regional real-time fusion, lightweight twin operation and rapid early warning, and the cloud is responsible for global data storage, deep model training, high-fidelity twin construction and complex inference.

[0054] The lightweight twin rendering and interaction unit uses WebGL and game engine optimization technology to achieve smooth visualization and interaction of large-scale scenes;

[0055] The secure and reliable mechanism unit enhances data access security authentication, transmission encryption, model robustness, system fault tolerance, and disaster recovery capabilities.

[0056] As a further aspect of the present invention: in the unified spatiotemporal reference engine, the time synchronization error is less than a preset threshold Δt, and the spatial coordinate transformation accuracy reaches the preset accuracy requirement.

[0057] As a further aspect of the present invention: in the fusion strategy unit driven by the waterway scene, the fusion weight calculation function f is trained and optimized by machine learning algorithm based on historical data and real-time scene data.

[0058] As a further aspect of the present invention: in the conflict resolution and data completion unit, the knowledge graph contains knowledge triples of waterway rules, ship characteristics, and historical accident patterns.

[0059] As a further aspect of the present invention: in the multi-scale fluid dynamics coupled model unit, the macroscopic hydrological model adopts a tidal harmonic analysis model, as shown in the following formula:

[0060]

[0061] Where h(t) is the water level at time t, h0 is the mean sea level, and f i H is the tidal junction factor. i σ is the amplitude of the tidal phase. i Let θ be the angular frequency of the tidal phase. i For the initial phase of the tide, u i K is the astronomical phase angle for tidal division. i Let θ be the lag angle of the tidal constituents, and n be the number of tidal constituents.

[0062] As a further aspect of the present invention: in the agent-driven ship behavior modeling unit, the autonomous agent also makes behavioral decisions based on the crew behavior pattern rule base, which is established through training on historical ship behavior data.

[0063] As a further aspect of the present invention: in the dynamic risk assessment and hierarchical early warning unit, the rule / knowledge graph-based reasoning engine uses production rules for risk assessment reasoning, and the production rule form is: IF<condition>THEN<conclusion>.

[0064] As a further aspect of the present invention: in the cloud-edge-device collaborative computing architecture unit, data transmission adopts an edge computing node caching and prefetching strategy to reduce data transmission latency.

[0065] As a further aspect of the present invention: a digital twin risk early warning method based on the fusion of multi-source data of waterways in the above system includes the following steps:

[0066] Data specific to the waterway are collected through a multi-source data acquisition module;

[0067] The collected source data are fused in a unified spatiotemporal coordinate system using a unified spatiotemporal reference engine; the fusion weights and confidence levels of each data source are dynamically adjusted by a fusion strategy unit driven by the waterway scenario based on different regions, meteorological conditions, and traffic density; and the conflict resolution and data completion unit uses knowledge graphs or deep learning models to identify and resolve sensor conflicts and perform high-precision predictive completion of missing data.

[0068] High-precision water flow simulation is achieved by fusing macroscopic hydrological models and microscopic CFD models through a multi-scale fluid dynamics coupling model unit; an agent-driven ship behavior modeling unit is used to build an autonomous agent for each ship, enabling it to make dynamic responses in the twin based on the environment and commands; and environmental element dynamic integration unit integrates meteorological, visibility, and water depth change environmental data in real time, and dynamically affects the ship's motion model and environmental performance in the twin.

[0069] The system utilizes a multimodal fusion risk feature extraction unit to extract deep-level risk correlation features through a spatiotemporal graph neural network or a multimodal Transformer model; combines a deep learning prediction model and a rule / knowledge graph-based inference engine to perform dynamic risk assessment through a dynamic risk assessment and graded early warning unit, outputting the risk level and specific risk type; and employs a risk tracing and root cause analysis unit to trace back and explain the causes of risk formation through explainable AI technology or inference paths.

[0070] Based on risk level, vessel type, location, and crew roles, the system intelligently pushes early warning information to different terminals through a multi-terminal intelligent adaptation and push unit, and provides differentiated operational suggestions. When a high-risk event is warned, the system simulates emergency scenarios on the digital twin through a digital twin sand table simulation unit, assessing the effectiveness of different emergency plans and potential chain risks. Actual handling instructions and results are fed back to the digital twin, and the early warning model and rule base are continuously optimized through the early warning-handling-feedback closed-loop unit.

[0071] The cloud-edge-device collaborative computing architecture unit designs a layered processing architecture to optimize data transmission and computing load distribution; the lightweight twin rendering and interaction unit uses WebGL and game engine optimization technologies to achieve smooth visualization and interaction of large-scale scenes; and the secure and trustworthy mechanism unit strengthens data access security authentication, transmission encryption, model robustness, system fault tolerance and disaster recovery capabilities.

[0072] The beneficial effects of this invention are:

[0073] Deep fusion of multi-source heterogeneous data unlocks data value: An innovative adaptive fusion algorithm is proposed for multi-source heterogeneous data unique to waterways, including AIS, radar, and video surveillance. Through a unified spatiotemporal benchmark engine, based on high-precision time synchronization technology and a geographic information system, the algorithm fully considers the impact of water flow and tides on ship positions, achieving precise spatiotemporal alignment of multi-source data. A waterway scenario-driven fusion strategy unit dynamically adjusts the fusion weights of data sources based on different regions, meteorological conditions, and traffic density, making data fusion more closely aligned with actual scenarios. A conflict resolution and data completion unit utilizes knowledge graphs and deep learning models to effectively solve data conflict and missing data problems. This series of mechanisms breaks down data fusion barriers, deeply integrates multi-source data, fully unlocks the value behind the data, and provides more comprehensive and accurate data support for waterway safety management.

[0074] A high-fidelity digital twin is constructed to recreate realistic waterway scenarios: The constructed digital twin goes beyond mere visualization, combining macroscopic hydrological models with microscopic CFD models through multi-scale fluid dynamics coupling model units. This enables high-precision water flow simulation in key areas such as bridge piers, wharves, and narrow waterways, realistically simulating the impact of water flow on ships. An agent-driven ship behavior modeling unit constructs an autonomous agent for each ship based on physical models and intelligent rules, allowing ships within the twin to make near-realistic dynamic responses based on the environment and commands, accurately reflecting ship maneuverability and collision avoidance behavior. A dynamic environmental element integration unit integrates real-time data on weather, visibility, and water depth changes, dynamically influencing the ship motion model to fully recreate the dynamic characteristics of the interaction between the waterway environment and ships. This digital twin provides a realistic and reliable virtual environment for waterway safety management, facilitating real-time monitoring and analysis by management personnel.

[0075] Intelligent risk warning and tracing enhances the scientific nature of decision-making: Abandoning traditional alarm methods based on simple thresholds, intelligent risk warning and tracing are achieved using deep learning and knowledge reasoning technologies. The multimodal fusion risk feature extraction unit utilizes spatiotemporal graph neural networks or multimodal Transformer models to jointly analyze data from various aspects such as ship dynamics, environmental conditions, waterway structure, and traffic conditions, extracting deep-level risk correlation features and accurately identifying potential risks. The dynamic risk assessment and graded warning unit combines deep learning prediction models and rule / knowledge graph-based inference engines to not only predict future ship trajectories and encounter situations but also conduct dynamic risk assessments based on rules and knowledge, outputting accurate risk levels and types. The risk tracing and root cause analysis unit uses explainable AI technology or reasoning path backtracking to deeply analyze the causes of risk formation, providing managers with detailed risk explanations. This enables managers to gain a deeper understanding of the nature of risks and formulate more targeted and scientific response measures.

[0076] Efficient virtual-real interaction and emergency management form a closed-loop optimization: This enables efficient and accurate delivery of early warning information from the digital twin virtual space to the physical waterway reality, and supports emergency response simulations and effect evaluations in the virtual space. The multi-terminal intelligent adaptation and push unit pushes early warning information in appropriate formats to shipboard terminals, mobile terminals, and shore-based monitoring screens based on factors such as risk level, vessel type, location, and crew roles, providing differentiated operational suggestions to ensure timely delivery of early warning information to relevant personnel. The digital twin sand table simulation unit supports operators in simulating various instructions and environmental changes on the digital twin during high-risk event warnings, allowing real-time evaluation of the effectiveness of different emergency plans and potential cascading risks, assisting in decision-making for the optimal response. The early warning-response-feedback closed-loop unit feeds back actual response instructions and results to the digital twin for continuous optimization of the early warning model and rule base, forming a management closed loop. This efficient virtual-real interaction and emergency management model significantly improves the efficiency and accuracy of waterway emergency management and enhances waterway safety assurance capabilities.

[0077] Optimizing the system architecture ensures high-performance operation: An innovative cloud-edge-device collaborative computing architecture is adopted. Sensor / ship-side nodes perform initial data filtering and lightweight processing, reducing data transmission pressure. Edge computing nodes are responsible for regional real-time fusion, lightweight twin operation, and rapid early warning, enabling localized data processing and improving response speed. The cloud is responsible for global data storage, deep model training, high-fidelity twin construction, and complex simulations, ensuring overall system operation and data management. Simultaneously, the lightweight twin rendering and interaction unit utilizes WebGL and game engine optimization technologies to achieve smooth visualization and interaction in large-scale scenes while ensuring clear presentation of key information, meeting the access needs of web or mobile devices. The security and trust mechanism unit strengthens data access security authentication, transmission encryption, model robustness, system fault tolerance, and disaster recovery capabilities, ensuring the system's real-time performance, scalability, and security in large-scale waterways and high-density ship scenarios, providing a solid guarantee for stable system operation. Attached Figure Description

[0078] The invention will now be further described with reference to the accompanying drawings.

[0079] Figure 1 This is a system block diagram of a digital twin risk early warning system for waterway multi-source data fusion according to the present invention. Detailed Implementation

[0080] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0081] According to the appendix Figure 1 A digital twin risk early warning system for waterway multi-source data fusion, comprising:

[0082] The multi-source data acquisition module is used to collect data of waterway-specific types, including AIS data, radar data, video surveillance data, hydrological and meteorological data, waterway mapping data, ship report data, shore-based sensor data, and UAV patrol data.

[0083] Specifically, for AIS data acquisition: a Comar AIS6500 VHF receiver was selected. This device conforms to the IEC62287 standard, supports AIS signal reception for Class A and Class B ships, and has a receiving sensitivity of -107dBm. The raw AIS data is transmitted to the data preprocessing unit via an RS-485 interface, and the AIVDM / AIVDO decoding protocol is used to extract information such as the ship's MMSI, latitude and longitude (accuracy ±2.5m), speed (accuracy ±0.1 knots), and heading (accuracy ±1°).

[0084] Specifically, radar data acquisition involves deploying a Raytheon Anschütz X-band radar, model XR5000, with a detection range of 0.1-72 nautical miles, a range resolution of 1 meter, and an azimuth resolution of 1.5°. Using the TCP / IP protocol, radar echo data (including target range, azimuth, and radial velocity) is transmitted to the system via UDP packets. A Kalman filter-based multi-target tracking algorithm (with the filter gain matrix dynamically adjusted according to the radar measurement noise covariance matrix) is employed to achieve stable tracking of ship targets.

[0085] Specifically, video surveillance data acquisition involves installing Hikvision DS-2CD8647FWDV2-IHS 4K cameras in key areas of the waterway. These cameras operate at 25fps, are equipped with 12mm focal length lenses, and offer a 120° field of view. The video stream is transmitted to edge computing nodes via the RTSP protocol. These nodes are equipped with NVIDIA Jetson Xavier NX development boards and run a YOLOv8 target detection model (using pre-trained COCO dataset weights, undergoing 1000 iterations of transfer learning for ship targets, achieving a detection accuracy of 0.85 mAP@0.5). This model identifies and tracks ship targets in real time, outputting information such as target location and size.

[0086] Specifically, hydrological and meteorological data acquisition: Hydrological data acquisition utilizes SonTek's RiverSurveyor M9ADCP device, operating at a frequency of 1.5MHz, measuring water depths from 0.1 to 30 meters, with a flow velocity measurement accuracy of ±0.01 m / s. The measured flow velocity profiles and flow direction data are transmitted to the system via an Ethernet interface. Meteorological data is acquired in real-time through the AccuWeather API interface, providing parameters such as wind speed, wind direction, visibility, and rainfall, with a data update frequency of once per minute.

[0087] Specifically, waterway surveying data acquisition involves importing electronic nautical chart data conforming to the S-57 standard, which is then regularly updated by a professional surveying agency. Real-time water depth measurements are performed using a DJI Matrice 300RTK drone equipped with a RIEGLVUX-1UAV LiDAR scanner, achieving a LiDAR point cloud density of 50 points / m². 2 The water depth measurement accuracy is ±0.1m. The scanned data is transmitted to the system via UAV ground station software and fused with electronic chart data to construct a dynamic three-dimensional waterway model.

[0088] Specifically, ship report data collection: Crew members manually report ship anomalies, cargo types, and other information via a customized "Smart Navigation" APP. The APP uses HTTPS protocol to communicate with the system server to ensure data transmission security. Simultaneously, AIS base stations receive ship arrival / departure reports (ETA / ETD) and other information, transmitting them to the system database via a dedicated data channel.

[0089] The multi-source data fusion processing module includes a spatiotemporal benchmark unification engine, a waterway scenario-driven fusion strategy unit, and a conflict resolution and data completion unit.

[0090] The unified spatiotemporal reference engine establishes a unified spatiotemporal coordinate system through the following formula, enabling spatiotemporal alignment of multi-source heterogeneous data:

[0091]

[0092] Among them, P adjusted For the adjusted ship position, P original The original ship position is given, t0 is the start time, t is the current time, and V is the current position. current (t) is the water flow velocity vector that varies with time; high-precision time synchronization technology is used to ensure time consistency of all data sources, and spatial data is converted to a unified coordinate system based on waterway GIS data;

[0093] Specifically, the time synchronization hardware configuration is as follows: The shipborne equipment integrates a Trimble BD992 GPS timing module, which has dual-frequency L1 / L2 reception capability and a time accuracy of ±1μs. Edge nodes and the cloud server are configured with Stratum Class 1 NTP servers (model SyncServerS600), and through network time protocol stack optimization (using symmetrical mode for time synchronization, with a synchronization period of 1 second), the system-wide time synchronization error is ensured to be <10ms. A timestamp accurate to milliseconds (format YYYY-MM-DDHH:MM:SS.SSS) is added to each piece of collected data. A cubic spline interpolation algorithm is used to align data of different frequencies (e.g., radar 10Hz, AIS 1-10Hz, video 25Hz), and a time mapping table is established to record the interpolated time correspondence.

[0094] Specifically, the spatial coordinate transformation algorithm is implemented as follows: Using the WGS84 geographic coordinate system as a unified benchmark, the radar polar coordinates (range, azimuth) and video pixel coordinates are converted to latitude and longitude using a seven-parameter transformation method (Bursha-Wolf model). In the specific calculation process, at least three control points with known precise coordinates (with centimeter-level accuracy) around the waterway are collected, and the seven parameters (three translation parameters, three rotation parameters, and one scale parameter) are solved using the least squares method. For the correction of the ship's position by the water flow, the real-time water flow velocity field obtained based on the FVCOM model is used. Using a time step of 100ms, the Euler method was employed through the formula... The corrected ship position is calculated, where the integral calculation is achieved by accumulating the displacement vector within each time step.

[0095] The fusion strategy unit, driven by the waterway scenario, dynamically adjusts the fusion weights w of each data source based on different regions, weather conditions, and traffic density using the following formula. i :

[0096] w i =f(S,C i )

[0097] Where S represents the current set of waterway scenario parameters, including region type, weather conditions, and traffic density, and C i Let f represent the set of feature parameters of the i-th data source, and f be the fusion weight calculation function;

[0098] Specifically, scene parameter quantification and encoding: A three-dimensional scene feature vector S = [R, W, T] is established, where the region type R adopts a one-hot encoding method (open water: [1,0,0,0,0,0], narrow waterway: [0,1,0,0,0,0], and so on); in the meteorological condition vector W, the visibility level is divided into 6 intervals (<0.1km = 0, 0.1-0.5km = 1, ..., >10km = 5), and the wind force level is divided into 13 levels (0-12) according to the Beaufort scale; the traffic density index T is calculated based on the number of ships per unit area and is mapped to the [0,1] interval using a linear piecewise function (low <5 ships / km). 2 Corresponding to 0, 5-20 ships / km 2 Linear mapping, height > 20 ships / km 2 Corresponding to 1).

[0099] Specifically, the training of the fusion weight calculation model involves using the Gradient Boosting Decision Tree (GBDT) algorithm to train the fusion weight function f(S, C). i The training dataset contains 1 million records of waterway operation data from the past year, covering multi-source data and corresponding optimal fusion results (determined through expert annotation and historical accident analysis) under different scenarios. During model training, the learning rate was set to 0.1, the number of iterations to 500, the maximum tree depth to 5, and the mean squared error (MSE) loss function. During system operation, the training data was updated hourly to retrain the model and adapt to changes in the waterway environment. For example, in scenarios with bridges, low visibility (W=2), and high traffic density (T=0.8), the model automatically assigned weights: radar 0.4, video surveillance 0.35, AIS 0.2, and other data sources 0.05, prioritizing high-precision local detection data.

[0100] The conflict resolution and data completion unit utilizes knowledge graphs or deep learning models for sensor conflict identification and resolution, and employs deep learning models such as Transformer to predictively complete missing data. The Transformer model based on the attention mechanism is calculated as follows:

[0101]

[0102] Where Q is the query vector, K is the key vector, V is the value vector, and d is the value vector. k The dimension of the key vector;

[0103] Specifically, the knowledge graph is constructed and reasoned as follows: By analyzing waterway management regulations such as the International Maritime Collision Prevention Regulations (IMDG) and the Inland Waterway Navigation Standards, ship technical manuals (such as maneuvering performance parameters for different ship types), and historical accident reports from the past five years, entity information is extracted, including: ships (attributes include MMSI, type, deadweight tonnage, length, and beam), waterways (attributes include area ID, depth, width, and radius of curvature), rules (attributes include collision prevention rule clauses, applicable scenarios, and consequences of violations), and accidents (attributes include time, type, cause, and damage). Relationship types such as "belongs to," "constraints," "caused by," and "affects" are defined to construct knowledge triples. The Neo4j graph database is used to store and manage the knowledge graph, configured with an 8-core CPU, 32GB of memory, and 1TB of SSD storage. When the position deviation between AIS and radar exceeds 50m, knowledge graph reasoning is triggered. Relevant rules and historical cases are retrieved using the Cypher query language. If the current scenario is an anchorage (where the ship may be stationary), the radar detection result is accepted, and the AIS data is marked as abnormal. This conflict case is then recorded in the knowledge graph for subsequent learning.

[0104] Specifically, the data completion algorithm was optimized as follows: a Transformer model with timestamps was used to process the sequence data. The model input consisted of a multi-source data time window from the past 10 minutes (including AIS, radar, video, etc., arranged in chronological order), and missing values ​​were masked using [MASK] tags. The model structure included 12 encoder layers, each consisting of a multi-head attention mechanism (8 heads) and a feedforward neural network. The training data came from a complete dataset spanning three months (500,000 sequences in total), with missing values ​​generated through random masking (masking rate of 15%). The model output was the completed continuous trajectory. During training, the Adam optimizer was used with an initial learning rate of 0.001, employing a cosine annealing learning rate adjustment strategy. The training period was 20 epochs, and the mean squared error (MSE) loss function was used. The final average completion error on the test set was less than 5m.

[0105] The digital twin construction module includes a multi-scale fluid dynamics coupled model unit, an agent-driven ship behavior modeling unit, and a dynamic integration unit for environmental elements.

[0106] A multi-scale fluid dynamics coupled model unit, integrating macroscopic hydrological models and microscopic CFD models, performs microscopic CFD calculations based on the Navier-Stokes equations at bridge piers, wharves, and narrow waterways. The formulas are as follows:

[0107]

[0108] Where ρ is the fluid density, v is the velocity vector, t is time, p is pressure, μ is dynamic viscosity, and f is external force; the macroscopic hydrological model and the microscopic CFD model are coupled through data transfer and boundary conditions to achieve high-precision water flow simulation.

[0109] Specifically, the macroscopic hydrological model employs FVCOM (Finite Volume Community Ocean Model) to simulate tides and ocean currents. The model's spatial resolution is set to 100m, with a time step of 60s. The horizontal grid uses an unstructured triangular mesh, and the vertical grid uses 10 layers of sigma coordinates. The model's boundary conditions are set as follows: the open boundary uses tidal data provided by a global tidal model (such as TPXO), and the closed boundary uses no-slip boundary conditions based on the actual shoreline topography. Parameters such as water level, flow velocity, and flow direction are synchronized to the digital twin in real time via a WebService interface (using the SOAP protocol), with a data update frequency of once per minute.

[0110] Specifically, the micro-CFD model: In key areas such as the bridge area, a 50m radius around the bridge piers was meshed with a minimum mesh size of 5m. The Reynolds-averaged Navier-Stokes (RANS) equations were solved using ANSYS Fluent software, and the k-ε turbulence model was selected. The computational domain inlet was set to a velocity inlet boundary condition (velocity provided by the macroscopic hydrological model), the outlet to a pressure outlet boundary condition, and the bridge pier surface to a no-slip wall boundary condition. The model used a second-order upwind scheme to discretize the convection terms, and the SIMPLE algorithm was used to solve the pressure-velocity coupled equations. Flow field data (including velocity, pressure, vorticity, etc.) was output every second, and local flow field changes were fed back to the macroscopic hydrological model via a data interface to achieve two-way coupling.

[0111] Specifically, the coupling interface design involves developing a custom data conversion interface to convert data such as water level and flow velocity output from the macroscopic hydrological model into the boundary condition format required by the microscopic CFD model. At the same time, it converts data such as flow velocity attenuation and pressure changes caused by flow around bridge piers calculated by the microscopic CFD model into a format that the macroscopic hydrological model can accept. Data exchange is achieved through a timed task (once every 10 seconds) to ensure the collaborative operation of multi-scale models.

[0112] The agent-driven ship behavior modeling unit constructs an autonomous agent for each ship based on a physical model and intelligent rules. The ship maneuvering physical model adopts the MMG (Mathematical Model Group) model, and its equations of motion are as follows:

[0113]

[0114] Where m is the ship's mass, u and v are the ship's longitudinal and lateral velocities respectively, r is the ship's bow angular velocity, and x... GFor the longitudinal coordinate of the ship's center of gravity, I z Let X be the moment of inertia of the ship about the z-axis, and let X, Y, and N be the longitudinal force, lateral force, and bow moment acting on the ship, respectively.

[0115] Specifically, MMG model parameterization involves establishing a ship parameter database to store detailed parameters for different ship types (bulk carriers, container ships, tankers, etc.), including mass m and moment of inertia I. z , centroid coordinates x G Square coefficient C B Etc. Ship parameters can be obtained through the ship's MMSI query. The database uses a MySQL relational database, configured with a master-slave replication architecture to improve data retrieval performance. In the calculation of external forces, the propeller thrust X... P =K P n 2 The thrust coefficient K P The speed *n* is determined using a lookup table based on the ship type and propeller model, and is collected in real-time by the ship's engine control system. Hydrodynamic derivatives are fitted using the ITTC formula, such as the lateral hydrodynamic coefficient *Y*. v =-0.5ρL 3 C B (ρ is the density of water, L is the length of the ship).

[0116] Specifically, collision avoidance rules are embedded: a COLREGs rule decision tree is established, where each node contains rule conditions (e.g., CPA < 500m and TCPA < 10min for both ships) and decision actions (e.g., full starboard, deceleration, etc.). When a ship is operating within the digital twin, the agent monitors the status of surrounding ships in real time and triggers rule inference. In a head-on encounter situation, a "full starboard" turning command is generated with a turning angle of 30°; in a crossover situation, the ship traveling in the straight maintains its course, while the ship giving way turns right to avoid a collision, with the turning angle calculated based on DCPA (expected encounter distance) (△ψ = arcsin(DCPA / (2V)). rel ·TCPA)) where V rel (Relative speeds of the two ships). The decision tree is trained and optimized using historical collision avoidance cases to continuously improve the accuracy of rule matching.

[0117] The dynamic integration unit for environmental elements integrates real-time environmental data on meteorological, visibility, and water depth changes, and dynamically influences the ship motion model through the following formula:

[0118]

[0119] in, This is the actual acceleration vector of the ship. The acceleration vector generated by the ship's own power. This represents the acceleration vector caused by environmental factors.

[0120] Specifically, it integrates real-time environmental data such as meteorological conditions (wind speed, wind direction, wave height), visibility, and water depth changes, and uses formulas to... The ship motion model is dynamically updated. This includes environmental acceleration. The calculations are as follows: Wind speed impact is calculated using a wind pressure coefficient model to determine the force exerted by the wind on the ship, thus obtaining wind-induced acceleration; Wave height impact is calculated using a ship seakeeping model to determine the acceleration caused by wave forces; Visibility impact is mainly reflected in adjusting the ship's safe speed and collision avoidance decisions; Water depth changes are determined by comparing the ship's draft with the real-time water depth, and when approaching the critical water depth, the ship's drag coefficient is increased, and the ship's speed is reduced. The system updates environmental data every second and recalculates the ship's motion state to ensure that the ship's behavior in the digital twin is consistent with the actual environment.

[0121] The risk warning and source tracing analysis module includes a multimodal fusion risk feature extraction unit, a dynamic risk assessment and graded early warning unit, and a risk source tracing and root cause analysis unit.

[0122] The multimodal fusion risk feature extraction unit utilizes a spatiotemporal graph neural network (ST-GNN) for risk feature extraction, where nodes v in the graph structure... i The feature update formula is as follows:

[0123]

[0124] in, For node v i The feature vector at layer l+1, where σ is the activation function and N(i) is the node v. i The set of neighboring nodes, W l Let b be the weight matrix of the l-th layer. l Let l be the bias vector of the l-th layer;

[0125] Specifically, the multimodal fusion risk feature extraction unit constructs a graph structure with ships as nodes and undirected edges at distances less than 1000m. Node attributes include ship position, speed, heading, draft, and type, and the edge weight is the reciprocal of the distance between the two ships. A spatiotemporal graph neural network (ST-GNN) is used to extract risk features. The network structure contains three spatiotemporal convolutional layers, and the calculation formula for each layer is as follows:

[0126] in Features of neighboring nodes Let W represent the historical features of the current node, σ be the ReLU activation function, and W be the W value. (l) and This is a trainable weight matrix. The model was trained using the Adam optimizer with a learning rate of 0.001 and a training period of 10 epochs. The loss function used was cross-entropy loss. The model was trained on a historical accident dataset, and the final model achieved an accuracy of 90% in extracting risk features.

[0127] The dynamic risk assessment and hierarchical early warning unit combines a deep learning prediction model and a rule / knowledge graph-based inference engine for dynamic risk assessment. The deep learning prediction model uses an LSTM network, and its cell state update formula is as follows:

[0128]

[0129] Among them, f t For the output of the forget gate, i t For input gate output, For candidate cell states, C t In cellular state, o t For output gate output, h t To be in a hidden state, W f W i W C W o Let b be the weight matrix. f b i b c b o It is the bias vector;

[0130] Specifically, the LSTM model takes into account the ship's trajectory data (latitude, longitude, speed, and heading) over the past 10 minutes and current environmental data (wind speed, visibility, and current speed). The model structure consists of two LSTM layers (128 neurons each) and one fully connected output layer. The output layer, after being normalized using the Softmax function, outputs the trajectory prediction (latitude and longitude) for the next 5 minutes, along with the probabilities of collision, grounding, and dragging anchor. The rule engine pre-stores over 300 channel-specific rules, such as "IF (ship position ∈ bridge area) AND (draft > channel depth - 0.5m) THEN trigger grounding warning." It employs a forward inference strategy; when real-time data meets the rule conditions, the corresponding warning is triggered and mapped to different risk levels based on the severity of the risk (high risk: probability > 0.8, red warning; medium risk: 0.5-0.8, yellow warning; low risk: < 0.5, blue warning).

[0131] The risk tracing and root cause analysis unit uses explainable AI technology or reasoning paths to trace and explain the causes of risk formation.

[0132] Specifically, SHAP (SHapley Additive Explanations) interpretable AI technology is used for risk attribution. When the system issues a risk warning, the SHAP value calculates the contribution of each input feature (such as ship speed, course deviation, and ambient wind speed) to the risk prediction result, and visually presents the causes of the risk through visualizations (such as bar charts and dependency graphs). Simultaneously, it combines knowledge graphs to backtrack the reasoning path, analyzing the root causes of the risk from the perspectives of rule matching and historical cases. For example, for a collision warning, SHAP values ​​reveal that ship speeding and sudden course changes are the main influencing factors. Further analysis using the knowledge graph determines that the ship violated the speed limit rules of a specific waterway, providing managers with detailed risk explanations and decision-making basis.

[0133] The virtual-real interaction module includes a multi-terminal intelligent adaptation and push unit, a digital twin sand table simulation unit, and an early warning-handling-feedback closed-loop unit.

[0134] The multi-terminal intelligent adaptation and push unit intelligently pushes early warning information to different terminals based on risk level, vessel type, location, and crew role, and provides differentiated operation suggestions;

[0135] Specifically, the shipboard ECDIS terminal uses the JRC JANUS-5000 electronic chart display and information system. When a high-risk warning is received (e.g., a high collision risk level), an audible and visual alarm is triggered (buzzer alarm level 85dB, red warning light flashing frequency 2Hz). The screen marks the collision risk area with a red polygon, and simultaneously broadcasts "Collision hazard 500m to the left front, recommended to turn right 15°" using voice synthesis technology (using iFlytek voice engine). The warning information is refreshed every 2 seconds until the risk is cleared.

[0136] Specifically, the shore-based large screen utilizes a large-screen display system composed of 55-inch LCD splicing screens with a resolution of 3840×2160. Based on GIS map visualization, it dynamically displays ship trajectories (using different colors to distinguish ship types) and risk heat maps (the darker the color, the higher the risk, generated through Gaussian kernel density estimation). Operators can view detailed data by clicking on ship icons, including fused location information with an accuracy of 0.5m, ship attributes, and real-time risk assessment results. Simultaneously, the large screen features a multi-window mode, capable of displaying monitoring images and early warning information statistical reports from different waterway areas at the same time.

[0137] Specifically, the crew app: A "Smart Navigation" app based on Android and iOS platforms will be developed. Low / medium risk warnings will be pushed via text messages, including a description of the risk type and operational suggestions. For example, a low-risk visibility reduction warning will send a message such as "Current visibility in the waters is low. Please turn on your navigation lights and navigate with caution." A medium-risk approaching a dangerous area warning will send a message such as "Your vessel is approaching a prohibited navigation area. Please immediately adjust your course to [specific degree] and reduce your speed to [specific knot]." The app uses a message queue mechanism (RabbitMQ) to process warning messages, ensuring that messages are not lost or duplicated under high concurrency. It also supports offline message push; crew members can view historical warning information after logging back into the app.

[0138] The digital twin sand table simulation unit simulates issuing instructions, adjusting environmental parameters, and simulating ship responses on the twin when a high-risk event is warned, in order to evaluate the effectiveness of different emergency plans and potential chain risks.

[0139] Specifically, the system uses the Unity3D game engine to build a digital twin sandbox. When a high-risk event warning is issued (such as an impending collision between two ships), operators can set virtual obstacles and adjust weather parameters (such as instantly increasing wind speed to 20 m / s and reducing visibility to 200 m) by dragging the mouse or entering parameters in the command input box. The system calculates the ship's response based on the ship's maneuverability physics model and collision avoidance rules in the agent-driven ship behavior modeling unit. For example, when the command to "force decelerate to 5 knots" is issued, the system calculates the ship's inertial sliding distance using the formula "sliding distance = 0.1 × ship deadweight tons × current speed²" based on parameters such as ship mass and current speed, and updates the ship's position in the digital twin in real time. It also displays the distance change curves with surrounding ships to intuitively demonstrate the effects of different emergency plans and potential chain risks, assisting operators in deciding on the optimal course of action. In addition, the sandbox simulation supports multi-user collaborative operation, allowing shore-based management personnel and ship operators to log in to the system simultaneously and discuss emergency strategies in a virtual environment.

[0140] The early warning-response-feedback closed-loop unit feeds back actual response instructions and results to the digital twin, optimizing the early warning model and rule base;

[0141] Specifically, after the emergency response instructions are actually executed, the response results data are collected through shipboard terminal equipment (such as ECDIS and AIS devices) and shore-based monitoring systems. For example, the shipboard ECDIS records the actual speed changes and course adjustments of the vessel in real time, while the AIS device provides feedback on whether the vessel has changed its navigation status according to the instructions; the shore-based monitoring system confirms the vessel's position and changes in the surrounding environment through radar and video surveillance. This result data is fed back to the digital twin via the encrypted WebSocket protocol. The system compares and analyzes the actual data with the predicted data in the digital twin, uses the backpropagation algorithm to adjust the parameters of the risk warning model and the vessel behavior model, and updates the relevant rules in the rule base. For example, it optimizes the COLREGs rule decision tree based on the actual collision avoidance situation, continuously optimizing the warning model and the rule base to form a management closed loop.

[0142] The system architecture optimization module includes a cloud-edge-device collaborative computing architecture unit, a lightweight twin rendering and interaction unit, and a secure and trustworthy mechanism unit.

[0143] The cloud-edge-device collaborative computing architecture unit is designed with a layered processing architecture. The sensor / ship end performs preliminary data filtering and lightweight processing, the edge computing node is responsible for regional real-time fusion, lightweight twin operation and rapid early warning, and the cloud is responsible for global data storage, deep model training, high-fidelity twin construction and complex inference.

[0144] Specifically, in terms of terminal layer equipment selection, the shipborne AIS equipment adopts the ComarAIS6500 model, which has data preprocessing capabilities and can filter invalid AIS packets (such as packets with checksum errors), with a preprocessing efficiency of over 95%. The camera edge nodes use NVIDIA Jetson Xavier NX development boards, running the YOLOv8 target detection model, achieving a detection frame rate of up to 15fps when processing 1080P video streams. The edge servers deployed at the port base station use Dell PowerEdge R750 processors, configured with 2×Intel Xeon Gold 6348 CPUs, 256GB of memory, and 4×NVIDIA A10 GPUs. They are responsible for data fusion within a 10km range, running a lightweight twin (a simplified fluid model with a grid resolution of 50m), and using a Redis cache database to store frequently accessed data (such as basic ship information and real-time location), enabling fast data reading and a warning response time of <200ms. The cloud-based system utilizes an AWS EKS (Elastic Kubernetes Service) cluster, configured with 50 m5.4xlarge instances as compute nodes. Storage is provided by Amazon S3 object storage, capable of storing three years of historical data. The ST-GNN model is retrained weekly in the cloud using AWS SageMaker machine learning service, employing a distributed training strategy to support concurrent simulations for 1000+ ships, meeting the needs of large-scale waterway monitoring.

[0145] The lightweight twin rendering and interaction unit uses WebGL and game engine optimization technology to achieve smooth visualization and interaction of large-scale scenes;

[0146] Specifically, web-based twin rendering is implemented using the Three.js engine, employing Level of Detail (LOD) technology to optimize rendering performance. For ships close to the viewpoint (<500m), a high-precision hull model (100,000+ triangles) is displayed, including details such as hull textures, sails, and funnels. For ships at medium distance (500-2000m), the model is simplified to medium precision (10,000-100,000 triangles), retaining the main hull structure. Ships at long distance (>2000m) are simplified to icons (<1000 triangles). Simultaneously, WebGL 2.0's off-screen rendering technology is utilized to pre-render scenes outside the current field of view to textures, quickly displaying them when switching scenes, ensuring a stable frame rate of 60FPS even in large-scale scenes displaying 1000 ships. Furthermore, interactive operations such as mouse wheel zoom, left-click panning, and right-click rotation are supported, allowing users to easily view the waterway scene from all angles.

[0147] The secure and reliable mechanism unit enhances data access security authentication, transmission encryption, model robustness, system fault tolerance, and disaster recovery capabilities.

[0148] Specifically, data access adopts the OAuth2.0 authentication protocol. Sensor / ship-side devices must authenticate using a client ID and client key. After successful authentication, an access token is obtained before data transmission. SSL / TLS1.3 encryption is used during data transmission to ensure data security over the public network. An Intrusion Detection System (IDS) using the Suricata open-source software is deployed to monitor network traffic in real time. It identifies network attack behaviors, such as DDoS attacks and SQL injection attacks, through rule matching (Snort rule sets) and anomaly detection (machine learning-based traffic pattern analysis). When an attack is detected, the attack source IP address is automatically blocked. System data is backed up regularly, using a combination of full backups (weekly) and incremental backups (hourly). Backup data is stored in an off-site data center to improve system fault tolerance and disaster recovery capabilities, ensuring stable system operation.

[0149] As a further aspect of the present invention: in the unified spatiotemporal reference engine, the time synchronization error is less than a preset threshold Δt, and the spatial coordinate transformation accuracy reaches the preset accuracy requirement.

[0150] Specifically, regarding time synchronization, the selected Trimble BD992 GPS timing module possesses high-precision time output capabilities and enhances signal reception through an external antenna, ensuring a time accuracy of ±1μs under various environments. The Stratum Level 1 NTP server (SyncServerS600) is configured with dual power supply redundancy and network redundancy to prevent single-point failures from affecting time synchronization. The system periodically (once per hour) calibrates the NTP server, adjusting its own time deviation by comparing it with an international atomic clock time source (such as the NIST time server). During network time protocol stack optimization, a symmetrical mode is used for time synchronization, with a synchronization period set to 1 second. An algorithm dynamically adjusts the time offset compensation amount to ensure that the overall system time synchronization error is less than a preset threshold Δt (in practical applications, Δt is set to 10ms). For ensuring spatial coordinate transformation accuracy, high-precision GIS data is used as the benchmark, with the data source being aerial photogrammetry results from professional surveying and mapping institutions, achieving centimeter-level accuracy. The system employs professional surveying-grade coordinate transformation algorithms, such as the Bursa-Wolf model. During the transformation process, at least three control points with known precise coordinates around the waterway are collected (control point coordinates are obtained through GPS RTK measurement, with a horizontal accuracy of ±2cm and an elevation accuracy of ±3cm). The seven parameters are solved using the least squares method to ensure that the coordinate transformation accuracy meets the preset requirements (e.g., horizontal accuracy <0.5m, elevation accuracy <0.3m). Simultaneously, the system periodically (once per quarter) re-measures the control points and updates the seven parameters to guarantee the long-term accuracy of the coordinate transformation.

[0151] As a further aspect of the present invention: in the fusion strategy unit driven by the waterway scene, the fusion weight calculation function f is trained and optimized by machine learning algorithm based on historical data and real-time scene data.

[0152] In the data collection phase, a distributed data acquisition system was established. Multiple data acquisition nodes were deployed along the waterway to continuously collect one year's worth of historical waterway operation data, covering multi-source data and corresponding optimal fusion results across different seasons, time periods, and weather conditions. The optimal fusion result was determined by organizing a review panel composed of waterway management experts and ship navigation experts to evaluate and score the multi-source data fusion schemes for each scenario, selecting the scheme with the highest score as the optimal result. A total of 1 million valid training samples were collected. Model training employed the Gradient Boosting Decision Tree (GBDT) algorithm. During training, a learning rate of 0.1 was set, optimized within the range of [0.01, 0.2] using a grid search method. The number of iterations was 500, determined by observing the model's loss curve on the validation set until the loss no longer significantly decreased. The maximum tree depth was 5 to prevent overfitting. The loss function used was Mean Squared Error (MSE). By minimizing the mean squared error between the predicted weights and the actual optimal weights, the model learned the mapping relationship between the data and the fused weights. During system operation, a data update task is triggered every hour to add the newly collected one-hour data to the training dataset and delete the oldest one-hour data to maintain a stable training dataset size. The model is then retrained so that the fusion weight calculation function f can be dynamically optimized based on the latest historical data and real-time scene data, adapting to the constantly changing waterway environment and ensuring accurate allocation of fusion weights from various data sources under different scenarios.

[0153] As a further aspect of the present invention: in the conflict resolution and data completion unit, the knowledge graph contains knowledge triples of waterway rules, ship characteristics, and historical accident patterns.

[0154] Specifically, in the knowledge extraction phase, a professional team conducted in-depth studies of more than 10 waterway management regulations, including the International Regulations for Preventing Collisions at Sea (ICP-5) and the Inland Waterway Navigation Standards, extracting information such as rule clauses, applicable scenarios, and violation handling. They also analyzed technical manuals for over 100 different ship types to obtain attributes such as ship maneuvering performance parameters and equipment configurations. Furthermore, they compiled over 2,000 historical accident reports from the past five years, extracting key information such as the time, location, type, and cause of the accidents. In the knowledge modeling phase, entity types were defined, including: ships (attributes include MMSI, type, deadweight tonnage, length, beam, draft, main engine power, etc.), waterways (attributes include area ID, name, depth, width, radius of curvature, waterway grade, etc.), rules (attributes include rule number, clause content, applicable scenarios, and consequences of violations, etc.), and accidents (attributes include accident number, time, type, cause, and extent of damage, etc.). The system defines relationship types such as "belongs to" (e.g., a vessel belongs to a certain waterway area), "constraints" (e.g., rules constrain vessel behavior), "caused by" (e.g., an accident is caused by certain factors), and "affects" (e.g., waterway conditions affect vessel navigation), constructing knowledge triples. In the knowledge storage phase, Neo4j graph database is used for storage and management, with the database server configured with an 8-core Intel Xeon Gold 6248 RCPU, 32GB of memory, and 1TB SSD storage. In database design, indexes are created to improve query efficiency, such as indexes for frequently queried fields like vessel MMSI, waterway area ID, and rule number. The knowledge graph is updated regularly (monthly), adding newly released waterway management regulations, technical parameters of newly built vessels, and newly occurring accident cases to ensure it accurately reflects the actual waterway conditions and provides reliable knowledge support for conflict resolution. Simultaneously, a knowledge graph visualization tool is developed to facilitate management personnel in viewing and maintaining the knowledge graph structure.

[0155] As a further aspect of the present invention: in the multi-scale fluid dynamics coupled model unit, the macroscopic hydrological model adopts a tidal harmonic analysis model, as shown in the following formula:

[0156]

[0157] Where h(t) is the water level at time t, h0 is the mean sea level, and f i H is the tidal junction factor. i σ is the amplitude of the tidal phase. i Let θ be the angular frequency of the tidal phase. i For the initial phase of the tide, u i K is the astronomical phase angle for tidal division. i Let θ be the lag angle of the tidal constituents, and n be the number of tidal constituents.

[0158] Specifically, when the macroscopic hydrological model in the multi-scale fluid dynamics coupled model unit adopts the tidal harmonic analysis model, in terms of data collection, we cooperated with the National Marine Information Center to obtain tidal observation data of the target channel area over the past 19 years. The data came from 10 tidal observation stations along the coast, with an observation frequency of once per hour, ensuring the integrity and accuracy of the data. In the tidal parameter calculation stage, based on the tidal harmonic analysis theory, the main tidal constituents (such as M2, S2, K1, O1, etc.) were determined, and the least squares method was used to fit the observation data. In the specific calculation process, the tidal observation data were arranged in chronological order, and observation equations were constructed. The intersection factor f of the tidal constituents was obtained by solving the linear equation system. i Amplitude H i Angular frequency σ i Initial phase θ i Astronomical phase angle u i and late angle K i To improve computational accuracy, an iterative algorithm is used for optimization. The calculation stops when the parameter change between two consecutive iterations is less than a preset threshold (e.g., amplitude change < 0.01m, phase change < 0.1°). The calculated parameters are then substituted into the formula.

[0159] Water level changes are calculated at 1-minute intervals. During model validation, one year of tidal observation data not used in parameter calculations is selected as the validation set. The root mean square error (RMSE) between the model-predicted water level and the actual observed water level is calculated. When the RMSE < 0.1m, the model accuracy is considered satisfactory. Finally, macroscopic tidal data support is provided for the digital twin system. The data is transmitted to the digital twin system once per minute via a WebService interface (using the SOAP protocol).

[0160] As a further aspect of the present invention: in the agent-driven ship behavior modeling unit, the autonomous agent also makes behavioral decisions based on the crew behavior pattern rule base, which is established through training on historical ship behavior data.

[0161] Specifically, in the agent-driven ship behavior modeling unit, the autonomous agent makes decisions based on a rule base of crew behavior patterns as follows: In the data collection phase, it collects ship AIS trajectory data (5 million records), VDR (ship data recorder) data (total data volume reaching 10TB), and crew operation logs (covering navigation records of 1000 ships) from the past three years. In the data preprocessing phase, the AIS trajectory data is denoised and smoothed, and outliers are removed using a median filter algorithm; the VDR data is parsed to extract ship maneuvering commands (such as rudder angle changes and speed adjustments), equipment status, and other information; the crew operation logs are analyzed to extract key operational events and time points. In the model training phase, a Hidden Markov Model (HMM) is used to mine crew operation patterns and behavioral habits in different scenarios. First, a state space (e.g., ship navigation state, turning state, speed change state, etc.) and an observation space (e.g., changes in ship position, speed, rudder angle, etc.) are defined. The Baum-Welch algorithm is used to train and optimize the model parameters (transition probability matrix, observation probability matrix, initial state probability vector) for 10 epochs. The completed crew behavior pattern rule base contains over 500 typical operation sequences for different ship types in more than 10 typical scenarios, such as entering and leaving port, collision avoidance, berthing and unberthing. When the ship is running in the digital twin, the autonomous agent monitors the ship's state and surrounding environment in real time, determines the current scenario type, and then matches the corresponding operation sequence from the rule base, prioritizing decisions that conform to the crew behavior pattern, making the ship's behavior closer to reality. Simultaneously, the rule base supports online updates; when new crew operation data is collected, the HMM model is retrained, and the rule base content is updated to improve the accuracy and adaptability of decision-making.

[0162] As a further aspect of the present invention: in the dynamic risk assessment and hierarchical early warning unit, the rule / knowledge graph-based reasoning engine uses production rules for risk assessment reasoning, and the form of the production rule is: IF<condition>THEN<conclusion>.

[0163] Specifically, in the dynamic risk assessment and graded early warning unit, when the rule / knowledge graph-based reasoning engine implements production rule reasoning, during the rule writing phase, a rule writing team composed of waterway management experts, maritime law enforcement personnel, and ship navigation experts is organized to convert over 300 waterway-specific rules into production rule forms of "IF <condition> THEN <conclusion>". For example, "IF (ship position ∈ bridge area) AND (draft > waterway depth - 0.5m) THEN trigger grounding warning", and "IF (both ships CPA < 200m) AND (TCPA ​​< 5min) AND (collision avoidance lights not activated) THEN collision risk level +1". Rule storage uses a MySQL database, with rule tables designed to store rule numbers, condition expressions, conclusions, risk levels, etc., and indexes are created to improve rule query efficiency. During the reasoning execution phase, when the system performs risk assessment, it matches real-time data such as ship position, draft, heading, CPA, and TCPA with the rule conditions. A forward reasoning strategy is employed, starting from known facts (real-time data) and matching them according to the conditions of the rules. When a condition is met, the corresponding conclusion is triggered, and an early warning operation is executed. During the reasoning process, the Rete algorithm is used to optimize rule matching efficiency. This algorithm reduces redundant matching calculations and improves reasoning speed by constructing a network structure to store the condition and conclusion information of the rules. When multiple rules match successfully, they are processed according to rule priority (pre-set, such as rules involving personal safety having the highest priority) to ensure timely and accurate issuance of early warning signals. These warnings are mapped to different risk levels based on the severity of the risk (high risk: probability > 0.8, red warning; medium risk: 0.5-0.8, yellow warning; low risk: < 0.5, blue warning), and displayed on the system interface using different colors and icons.

[0164] As a further aspect of the present invention: in the cloud-edge-device collaborative computing architecture unit, data transmission adopts an edge computing node caching and prefetching strategy to reduce data transmission latency.

[0165] In the cloud-edge-device collaborative computing architecture unit, data transmission employs edge computing node caching and prefetching strategies as follows: For caching, a data cache is set up on the edge computing node, using a Redis in-memory database as the caching medium, with 16GB of memory configured for data storage. The cached data uses an LRU (Least Recently Used) eviction policy; when the cache is full, the least recently used data is automatically deleted. The cached data types include multi-source data (AIS data, radar data, video surveillance data, etc.) from the last 10 minutes, as well as some commonly used basic navigation data (such as commonly used areas in electronic chart data and ship static information). Regarding the prefetching strategy, historical data is analyzed, and time series prediction algorithms (such as the ARIMA model) are used to predict data that may be needed in the future. In practice, a prediction model is established for each type of data, and model parameters (such as the p, d, and q values ​​in ARIMA(p,d,q)) are determined based on the time series characteristics of the data. When the data request frequency reaches a certain threshold (e.g., the request frequency for ship data in a certain area exceeds 10 times / minute in the past 5 minutes), relevant data is prefetched from the cloud to the edge node cache. For example, during peak periods for ship arrivals and departures (7-9 am and 5-7 pm daily, determined by historical data), AIS data and waterway mapping data of ships within a 5km radius of the port are pre-fetched. When a new request is received, the data is read directly from the cache, reducing data transmission latency. Actual testing showed that during peak periods, the data request response time was reduced from an average of 1.2 seconds to 0.3 seconds, improving system response speed and ensuring efficient system operation.

[0166] As a further aspect of the present invention: a digital twin risk early warning method based on the fusion of multi-source data of waterways in the above system includes the following steps:

[0167] Data specific to the waterway are collected through a multi-source data acquisition module;

[0168] The collected source data are fused in a unified spatiotemporal coordinate system using a unified spatiotemporal reference engine; the fusion weights and confidence levels of each data source are dynamically adjusted by a fusion strategy unit driven by the waterway scenario based on different regions, meteorological conditions, and traffic density; and the conflict resolution and data completion unit uses knowledge graphs or deep learning models to identify and resolve sensor conflicts and perform high-precision predictive completion of missing data.

[0169] High-precision water flow simulation is achieved by fusing macroscopic hydrological models and microscopic CFD models through a multi-scale fluid dynamics coupling model unit; an agent-driven ship behavior modeling unit is used to build an autonomous agent for each ship, enabling it to make dynamic responses in the twin based on the environment and commands; and environmental element dynamic integration unit integrates meteorological, visibility, and water depth change environmental data in real time, and dynamically affects the ship's motion model and environmental performance in the twin.

[0170] The system utilizes a multimodal fusion risk feature extraction unit to extract deep-level risk correlation features through a spatiotemporal graph neural network or a multimodal Transformer model; combines a deep learning prediction model and a rule / knowledge graph-based inference engine to perform dynamic risk assessment through a dynamic risk assessment and graded early warning unit, outputting the risk level and specific risk type; and employs a risk tracing and root cause analysis unit to trace back and explain the causes of risk formation through explainable AI technology or inference paths.

[0171] Based on risk level, vessel type, location, and crew roles, the system intelligently pushes early warning information to different terminals through a multi-terminal intelligent adaptation and push unit, and provides differentiated operational suggestions. When a high-risk event is warned, the system simulates emergency scenarios on the digital twin through a digital twin sand table simulation unit, assessing the effectiveness of different emergency plans and potential chain risks. Actual handling instructions and results are fed back to the digital twin, and the early warning model and rule base are continuously optimized through the early warning-handling-feedback closed-loop unit.

[0172] The cloud-edge-device collaborative computing architecture unit designs a layered processing architecture to optimize data transmission and computing load distribution; the lightweight twin rendering and interaction unit uses WebGL and game engine optimization technologies to achieve smooth visualization and interaction of large-scale scenes; and the secure and trustworthy mechanism unit strengthens data access security authentication, transmission encryption, model robustness, system fault tolerance and disaster recovery capabilities.

[0173] It should be stated that all data collected in this application was collected with the user's consent and authorization, and the uses of the data are legal and compliant, with the use and processing of the data complying with the relevant laws, regulations, and standards of the relevant regions. The above formulas are all dimensionless numerical calculations, derived from software simulations using collected large amounts of data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas were set by those skilled in the art based on the actual situation.

[0174] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A digital twin risk early warning system for waterway multi-source data fusion, characterized in that: include: The multi-source data acquisition module is used to collect waterway-specific data types, including AIS data, radar data, video surveillance data, hydrological and meteorological data, waterway mapping data, ship report data, shore-based sensor data, and UAV patrol data. The multi-source data fusion processing module includes a spatiotemporal benchmark unification engine, a channel scenario-driven fusion strategy unit, and a conflict resolution and data completion unit. The unified spatiotemporal reference engine establishes a unified spatiotemporal coordinate system through the following formula, thereby achieving spatiotemporal alignment of multi-source heterogeneous data: Among them, P adjusted For the adjusted ship position, P original The original ship position is given, t0 is the start time, t is the current time, and V is the current position. current (t) is the water flow velocity vector that varies with time; high-precision time synchronization technology is used to ensure time consistency of all data sources, and spatial data is converted to a unified coordinate system based on waterway GIS data; The fusion strategy unit driven by the waterway scenario dynamically adjusts the fusion weights w of each data source according to different regions, weather conditions, and traffic density using the following formula. i : w i =f(S,C i ) Where S represents the current set of waterway scenario parameters, including region type, weather conditions, and traffic density, and C i Let f represent the set of feature parameters of the i-th data source, and f be the fusion weight calculation function; The conflict resolution and data completion unit utilizes knowledge graphs or deep learning models to identify and resolve sensor conflicts, and employs deep learning models such as Transformer to predictively complete missing data. The Transformer model based on the attention mechanism is calculated as follows: Where Q is the query vector, K is the key vector, V is the value vector, and d is the value vector. k The dimension of the key vector; The digital twin construction module includes a multi-scale fluid dynamics coupling model unit, an agent-driven ship behavior modeling unit, and an environmental element dynamic integration unit. A multi-scale fluid dynamics coupled model unit, integrating macroscopic hydrological models and microscopic CFD models, performs microscopic CFD calculations based on the Navier-Stokes equations at bridge piers, wharves, and narrow waterways. The formulas are as follows: Where ρ is the fluid density, v is the velocity vector, t is time, p is pressure, μ is dynamic viscosity, and f is external force; the macroscopic hydrological model and the microscopic CFD model are coupled through data transfer and boundary conditions to achieve high-precision water flow simulation. The agent-driven ship behavior modeling unit constructs an autonomous agent for each ship based on a physical model and intelligent rules. The ship maneuvering physical model adopts the MMG (Mathematical Model Group) model, and its equations of motion are as follows: Where m is the ship's mass, u and v are the ship's longitudinal and lateral velocities respectively, r is the ship's bow angular velocity, and x... G For the longitudinal coordinate of the ship's center of gravity, I z Let X be the moment of inertia of the ship about the z-axis, and let X, Y, and N be the longitudinal force, lateral force, and bow moment acting on the ship, respectively. The dynamic integration unit for environmental elements integrates real-time environmental data on meteorological, visibility, and water depth changes, and dynamically influences the ship motion model through the following formula: in, This is the actual acceleration vector of the ship. The acceleration vector generated by the ship's own power. This represents the acceleration vector caused by environmental factors. The risk warning and source tracing analysis module includes a multimodal fusion risk feature extraction unit, a dynamic risk assessment and graded early warning unit, and a risk source tracing and root cause analysis unit. The multimodal fusion risk feature extraction unit utilizes a spatiotemporal graph neural network (ST-GNN) for risk feature extraction, where nodes v in the graph structure... i The feature update formula is as follows: in, For node v i The feature vector at layer l+1, where σ is the activation function and N(i) is the node v. i The set of neighboring nodes, W l Let be the weight matrix of the l-th layer. b l Let l be the bias vector of the l-th layer; The dynamic risk assessment and hierarchical early warning unit combines a deep learning prediction model and a rule / knowledge graph-based inference engine for dynamic risk assessment. The deep learning prediction model uses an LSTM network, and its cell state update formula is as follows: Among them, f t For the output of the forget gate, i t For input gate output, For candidate cell states, C t In cellular state, o t For output gate output, h t To be in a hidden state, W f W i W c W o Let b be the weight matrix. f b i b C b o It is the bias vector; The risk tracing and root cause analysis unit uses explainable AI technology or reasoning paths to trace and explain the causes of risk formation. The virtual-real interaction module includes a multi-terminal intelligent adaptation and push unit, a digital twin sand table simulation unit, and an early warning-handling-feedback closed-loop unit. The multi-terminal intelligent adaptation and push unit intelligently pushes early warning information to different terminals based on risk level, vessel type, location, and crew role, and provides differentiated operation suggestions; The digital twin sand table simulation unit simulates issuing instructions, adjusting environmental parameters, and simulating ship responses on the twin when a high-risk event is warned, in order to evaluate the effectiveness of different emergency plans and potential chain risks. The early warning-response-feedback closed-loop unit feeds back actual response instructions and results to the digital twin, optimizing the early warning model and rule base; The system architecture optimization module includes a cloud-edge-device collaborative computing architecture unit, a lightweight twin rendering and interaction unit, and a secure and reliable mechanism unit. The cloud-edge-device collaborative computing architecture unit is designed with a layered processing architecture. The sensor / ship end performs preliminary data filtering and lightweight processing, the edge computing node is responsible for regional real-time fusion, lightweight twin operation and rapid early warning, and the cloud is responsible for global data storage, deep model training, high-fidelity twin construction and complex inference. The lightweight twin rendering and interaction unit uses WebGL and game engine optimization technology to achieve smooth visualization and interaction of large-scale scenes; The secure and reliable mechanism unit enhances data access security authentication, transmission encryption, model robustness, system fault tolerance, and disaster recovery capabilities.

2. The digital twin risk early warning system for waterway multi-source data fusion according to claim 1, characterized in that: In the unified spatiotemporal reference engine, the time synchronization error is less than a preset threshold Δt, and the spatial coordinate transformation accuracy meets the preset accuracy requirements.

3. The digital twin risk early warning system for waterway multi-source data fusion according to claim 1, characterized in that: In the channel scene-driven fusion strategy unit, the fusion weight calculation function f is trained and optimized using machine learning algorithms based on historical data and real-time scene data.

4. The digital twin risk early warning system for waterway multi-source data fusion according to claim 1, characterized in that: In the conflict resolution and data completion unit, the knowledge graph contains knowledge triples related to waterway rules, ship characteristics, and historical accident patterns.

5. A digital twin risk early warning system for waterway multi-source data fusion according to claim 1, characterized in that: In the multi-scale fluid dynamics coupled model unit, the macroscopic hydrological model adopts the tidal harmonic analysis model, as shown in the following formula: Where h(t) is the water level at time t, h0 is the mean sea level, and f i H is the tidal junction factor. i σ is the amplitude of the tidal phase. i Let θ be the angular frequency of the tidal phase. i For the initial phase of the tide, u i K is the astronomical phase angle for tidal division. i Let θ be the lag angle of the tidal constituents, and n be the number of tidal constituents.

6. The digital twin risk early warning system for waterway multi-source data fusion according to claim 1, characterized in that: In the agent-driven ship behavior modeling unit, the autonomous agent also makes behavioral decisions based on the crew behavior pattern rule base, which is established through training on historical ship behavior data.

7. A digital twin risk early warning system for waterway multi-source data fusion according to claim 1, characterized in that: In the dynamic risk assessment and hierarchical early warning unit, the rule / knowledge graph-based reasoning engine uses production rules for risk assessment reasoning. The production rule form is: IF<condition>THEN<conclusion>.

8. A digital twin risk early warning system for waterway multi-source data fusion according to claim 1, characterized in that: In the cloud-edge-device collaborative computing architecture unit, data transmission adopts edge computing node caching and prefetching strategies to reduce data transmission latency.

9. A digital twin risk early warning method for waterway multi-source data fusion based on the system described in any one of claims 1-8, characterized in that, Includes the following steps: Data specific to the waterway are collected through a multi-source data acquisition module; The collected source data are fused in a unified spatiotemporal coordinate system using a unified spatiotemporal reference engine; the fusion weights and confidence levels of each data source are dynamically adjusted by a fusion strategy unit driven by the waterway scenario based on different regions, meteorological conditions, and traffic density; and the conflict resolution and data completion unit uses knowledge graphs or deep learning models to identify and resolve sensor conflicts and perform high-precision predictive completion of missing data. High-precision water flow simulation is achieved by fusing macroscopic hydrological models and microscopic CFD models through a multi-scale fluid dynamics coupling model unit; an agent-driven ship behavior modeling unit is used to build an autonomous agent for each ship, enabling it to make dynamic responses in the twin based on the environment and commands; and environmental element dynamic integration unit integrates meteorological, visibility, and water depth change environmental data in real time, and dynamically affects the ship's motion model and environmental performance in the twin. The system utilizes a multimodal fusion risk feature extraction unit to extract deep-level risk correlation features through a spatiotemporal graph neural network or a multimodal Transformer model. Combining a deep learning prediction model and a rule / knowledge graph-based inference engine, the system performs dynamic risk assessment through a dynamic risk assessment and hierarchical early warning unit, outputting the risk level and specific risk type. By using the risk tracing and root cause analysis unit, the causes of risk formation can be traced back and explained through explainable AI technology or reasoning paths. Based on risk level, vessel type, location, and crew role, the system intelligently pushes early warning information to different terminals through a multi-terminal intelligent adaptation and push unit, and provides differentiated operation suggestions. When a high-risk event is warned, emergency scenarios are simulated on the digital twin using a digital twin sand table simulation unit to evaluate the effectiveness of different emergency plans and potential chain risks; actual handling instructions and results are fed back to the digital twin, and the warning model and rule base are continuously optimized through the warning-handling-feedback closed loop unit; The cloud-edge-device collaborative computing architecture unit designs a layered processing architecture to optimize data transmission and computing load distribution; the lightweight twin rendering and interaction unit uses WebGL and game engine optimization technologies to achieve smooth visualization and interaction of large-scale scenes; and the secure and trustworthy mechanism unit strengthens data access security authentication, transmission encryption, model robustness, system fault tolerance and disaster recovery capabilities.

Images