Intelligent unmanned ship real-time state monitoring and situation prediction system

Abstract

An intelligent unmanned ship real-time state monitoring and situation prediction system is characterized in that a multi-source sensor module is connected to an embedded main control unit through a CAN bus or an Ethernet interface; the embedded main control unit is connected with an edge calculation module through a high-speed serial interface; the data processing module is used for packaging original sensing data and transmitting the data to the edge calculation module for local preprocessing, the edge calculation module is connected with the communication relay module through a 4G / 5G or satellite communication link, and state data after compression and feature extraction are uploaded to the shore-based situation fusion center. According to the invention, through three-dimensional deployment and tight coupling fusion of a multi-source heterogeneous sensor, an all-medium environment cognitive ability covering an air, water surface and underwater three-dimensional space is constructed, an inertial measurement unit and a global navigation satellite system receiver adopt hardware-level tight coupling packaging, and microsecond-level timestamp alignment and original observation value synchronization are realized by an onboard FPGA (Field Programmable Gate Array). The continuity and robustness of dynamic high-precision positioning are improved, and the accumulative error of the pure inertial navigation system is effectively inhibited.

Description

Technical Field

[0001] This invention belongs to the field of unmanned surface vessel (USV) monitoring technology, specifically an intelligent real-time status monitoring and situation prediction system for USVs. Background Technology

[0002] With the rapid growth in demand for autonomous and intelligent surface platforms from marine economic development, maritime security, environmental monitoring, and emergency rescue missions, the autonomous navigation capability, environmental adaptability, and mission reliability of unmanned surface vessels (USVs) have increasingly become the core focus of technological development. However, existing USV systems still face a series of technical challenges in the complex, dynamic, and highly uncertain marine environment:

[0003] First, most current unmanned surface vessels rely on single or loosely coupled sensor combinations, such as using only GNSS+IMU for positioning and supplementing with a single type of radar for obstacle detection. This architecture has significant flaws: On the one hand, GNSS signals are easily interrupted in island and reef shielded areas, under bridges, or in environments with strong electromagnetic interference, while traditional IMUs, lacking high-frequency synchronization and tight coupling processing mechanisms, experience significant drift in a short time, leading to inaccurate pose calculations. On the other hand, although lidar can provide high-precision point clouds, its performance drops sharply under rain, fog, splash, or low-light conditions, while millimeter-wave radar, although capable of all-weather operation, struggles to provide accurate geometric contours. If the two are not fused in a unified spatiotemporal coordinate system, it will lead to false detections, missed detections, or coordinate misalignment of targets.

[0004] Secondly, most existing unmanned surface vessels (USVs) use general-purpose embedded platforms, where the main control unit and computing unit are often interconnected via low-speed serial buses (such as UART, CAN, or USB). This makes it difficult to handle concurrent data streams from lidar point clouds, high-sampling-rate IMUs, and high-bandwidth multi-channel hydrophones, resulting in transmission delays and even frame drops. Furthermore, due to the lack of efficient local intelligent preprocessing capabilities, raw data is often uploaded without compression or feature extraction, wasting limited wireless communication resources. In the wide-area marine environment, cellular network coverage is sparse and unstable, with most systems relying on a single communication link (such as 4G). Once the signal is interrupted, shore-based communication is completely lost, making status feedback and remote intervention impossible, thus weakening mission continuity and safety.

[0005] Furthermore, at the level of situation prediction and decision support, existing unmanned surface vessel (USV) trajectory predictions are mostly based on simplified kinematic models (such as constant velocity or acceleration assumptions), failing to fully integrate real-time ocean weather disturbances (such as wind, waves, and currents) and the interactive effects of surrounding dynamic targets. This leads to prediction results that deviate significantly from reality under complex sea conditions. Even when some studies attempt to introduce machine learning methods, their training data is often limited to finite real navigation logs, lacking coverage of extreme or rare scenarios (such as sea state 9 or dense fishing vessel convoys), and failing to consider sensor noise, data gaps, and other real-world interference factors. Consequently, the models exhibit weak generalization ability and poor robustness. Summary of the Invention

[0006] In order to overcome the shortcomings of the prior art, the present invention provides an intelligent unmanned surface vessel real-time status monitoring and situation prediction system to at least partially solve the above-mentioned technical problems.

[0007] The technical solution adopted in this invention is as follows:

[0008] This invention proposes an intelligent unmanned surface vessel (USV) real-time status monitoring and situation prediction system, comprising:

[0009] Multi-source sensor modules, embedded main control units, edge computing modules, communication relay modules, and shore-based situation fusion centers are installed on the hull of the unmanned surface vessel.

[0010] The multi-source sensor module includes an inertial measurement unit, a global navigation satellite system receiver, a lidar, a millimeter-wave radar, a hydrophone array, and environmental parameter sensors. Each sensor is connected to the embedded main control unit via a CAN bus or an Ethernet interface.

[0011] The embedded main control unit is connected to the edge computing module through a high-speed serial interface, and is used to package the raw sensing data and transmit it to the edge computing module for local preprocessing.

[0012] The edge computing module connects to the communication relay module via a 4G / 5G or satellite communication link and uploads the compressed and feature-extracted status data to the shore-based situation fusion center.

[0013] The shore-based situation fusion center is equipped with a deep temporal neural network model, which is used to generate motion situation prediction results within a predetermined time window based on historical state sequences and current multidimensional inputs.

[0014] In one embodiment of the present invention, the inertial measurement unit and the global navigation satellite system receiver are integrated in the same package in a tightly coupled manner, and the original IMU data and the GNSS pseudorange and carrier phase observations are synchronized and timestamped through an onboard FPGA chip, wherein the IMU sampling frequency is not less than 200Hz and the GNSS receiver output frequency is not less than 10Hz.

[0015] In one embodiment of the present invention, the lidar and the millimeter-wave radar are respectively installed on the top of the bow mast and the side support of the unmanned surface vessel. Their fields of view overlap by at least 30 degrees in the horizontal plane. The lidar output point cloud data frame and the millimeter-wave radar target track list are coordinate unified by a spatiotemporal registration module in the embedded main control unit. The module performs dynamic rotation and translation transformation on the radar's original coordinate system based on the unmanned surface vessel's current roll angle, pitch angle, and heading angle.

[0016] In one embodiment of the present invention, the hydrophone array consists of four omnidirectional hydrophones forming a cross-shaped topology, positioned below the keel of the unmanned surface vessel at a depth of 0.8-1.2 meters above the water surface. The analog signals output by each hydrophone are pre-amplified by a low-noise amplifier and then connected to a multi-channel synchronous analog-to-digital converter with a sampling rate of not less than 96kHz. The signals are then transmitted to the embedded main control unit via an SPI bus. The embedded main control unit has a built-in direction-of-arrival estimation algorithm module that uses the phase difference of the signals received by the four-element array to construct a covariance matrix, and then calculates the azimuth and pitch angles of the underwater sound source.

[0017] In one embodiment of the present invention, the edge computing module adopts a heterogeneous computing architecture, including an ARM Cortex-A72 multi-core processor and an NPU neural network acceleration unit, wherein the ARM processor runs a lightweight Linux system and deploys a data compression algorithm to perform lossless compression based on LZ4 or Zstandard on the raw data from multiple sources sensors.

[0018] In one embodiment of the present invention, the communication relay module includes a dual-mode communication unit, which integrates a 4G / 5G cellular communication chip and an L-band satellite modem, respectively, and the two are dynamically switched by a link selection controller in the embedded main control unit.

[0019] In one embodiment of the present invention, the shore-based situation fusion center is deployed on a cloud server cluster. Its input interface receives compressed status data streams from multiple unmanned surface vessels and buffers and distributes them through a message queue mechanism. The deep temporal neural network model adopts an encoder and decoder structure. The encoder is composed of three layers of bidirectional GRU units stacked together to extract long-term dependencies of historical trajectories. The decoder is composed of two layers of attention-enhanced LSTM units, which combine the current environmental wind speed, ocean current vector and the position of nearby targets as external context inputs, and output a predicted sequence of position, velocity and heading updated at a frequency of 1Hz within the next 30 seconds.

[0020] In one embodiment of the present invention, the training dataset of the deep temporal neural network model includes real navigation logs and simulation-enhanced samples. The simulation samples are generated by a high-fidelity ocean dynamics model, covering different sea state levels, wind angles and obstacle densities. An adversarial perturbation mechanism is introduced during the training process, injecting random noise conforming to a Gaussian process distribution into the input sequence.

[0021] In one embodiment of the present invention, the embedded main control unit and the edge computing module are interconnected using a PCIeGen2x4 high-speed interface, with a physical layer transmission rate of not less than 2GB / s, and a double-buffered circular queue mechanism is established between them.

[0022] In one embodiment of the present invention, the environmental parameter sensors include a three-axis ultrasonic anemometer, a digital barometer, a seawater temperature / salinity probe, and a six-degree-of-freedom wave spectrometer, which are respectively installed on the top of the superstructure of the unmanned surface vessel, in the middle of the deck, on the hull outside the hull, and below the waterline of the bilge; the output signals of each sensor are connected to the dedicated ADC channel of the embedded main control unit after being isolated and amplified by a circuit, and the sampling period is uniformly triggered by the internal timer of the main control unit.

[0023] The beneficial effects of the technical solution of this invention are as follows:

[0024] This invention constructs a comprehensive environmental awareness capability covering three-dimensional space (air, surface, and underwater) through the three-dimensional deployment and tight coupling fusion of multi-source heterogeneous sensors. The inertial measurement unit (IMU) and the global navigation satellite system (GNSS) receiver are tightly coupled in hardware, and the onboard FPGA achieves microsecond-level timestamp alignment and synchronization with the original observation values. This not only improves the continuity and robustness of dynamic high-precision positioning, but also maintains short-term high-fidelity pose calculation through high-frequency IMU data in GNSS denial scenarios, effectively suppressing the cumulative error of the pure inertial navigation system. The lidar and millimeter-wave radar form at least 30 degrees of field of view overlap in space, and through the spatiotemporal registration module in the embedded main control unit, the original coordinate system is dynamically rotated and translated based on real-time roll, pitch, and heading angles, unifying the point cloud and track to the geographic ENU coordinate system. This achieves complementary advantages of optical and electromagnetic sensing, with the former providing high-resolution geometric details and the latter ensuring all-weather target detection capability. The fusion of the two improves the accuracy and anti-interference of obstacle recognition.

[0025] This invention achieves low-latency, high-efficiency preprocessing of massive amounts of raw data through a high-speed interconnect architecture and heterogeneous edge computing. The embedded main control unit, as the data aggregation hub, not only completes the time alignment and format standardization of multiple sensors, but also establishes a double-buffered circular queue mechanism with the edge computing module through the PCIeGen2x4 interface to eliminate data transmission bottlenecks and ensure lossless pipeline transmission of high-bandwidth sources such as IMU, LiDAR, and hydrophone. The edge computing module adopts an ARM+NPU heterogeneous architecture. The ARM processor runs a lightweight Linux and performs LZ4 / Zstandard lossless compression, compressing the raw data volume by more than 50%. The NPU loads a quantized lightweight neural network model to extract key features such as acceleration mutations and obstacle approach trends in real time.

[0026] This invention ensures reliable uploading of status data in a wide-area marine environment through a dual-mode adaptive link switching mechanism. The 4G / 5G and L-band satellite communication units are dynamically managed by a link selection controller, which automatically switches based on signal strength and packet loss rate. Link identifiers and redundancy check fields are embedded in the data packets, supporting seamless reassembly of multipath data streams at the shore-based end.

[0027] This invention captures long-term dependencies in historical trajectories using a three-layer bidirectional GRU encoder, and fuses current wind speed, ocean currents, and external context of nearby targets with an attention-enhanced LSTM decoder to output a predicted sequence updated at 1Hz for the next 30 seconds. Simultaneously, its training set integrates real navigation logs and high-fidelity ocean dynamics simulation samples, and introduces adversarial perturbations conforming to a Gaussian process distribution, enabling the model to maintain stable output even in the face of sensor noise, missing data, or extreme sea conditions.

[0028] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0029] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0030] Figure 1 This is a module framework diagram of the intelligent unmanned surface vessel real-time status monitoring and situation prediction system proposed in this embodiment of the invention;

[0031] Figure 2 This is a first flowchart of the intelligent unmanned surface vessel real-time status monitoring and situation prediction system proposed in this embodiment of the invention;

[0032] Figure 3 This is a second flowchart of the intelligent unmanned surface vessel real-time status monitoring and situation prediction system proposed in this embodiment of the invention;

[0033] Figure 4 This is the third flowchart of the intelligent unmanned surface vessel real-time status monitoring and situation prediction system proposed in this embodiment of the invention. Detailed Implementation

[0034] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0035] The following describes an intelligent unmanned surface vessel real-time status monitoring and situation prediction system according to an embodiment of the present invention, with reference to the accompanying drawings.

[0036] like Figures 1 to 4 As shown, this embodiment of the invention provides an intelligent unmanned surface vessel (USV) real-time status monitoring and situation prediction system, including: a multi-source sensor module, an embedded main control unit, an edge computing module, a communication relay module, and a shore-based situation fusion center installed on the USV hull;

[0037] The multi-source sensor module includes an inertial measurement unit, a global navigation satellite system receiver, a lidar, a millimeter-wave radar, a hydrophone array, and environmental parameter sensors. Each sensor is connected to the embedded main control unit via a CAN bus or an Ethernet interface. The embedded main control unit is connected to the edge computing module via a high-speed serial interface to package the raw sensor data and transmit it to the edge computing module for local preprocessing.

[0038] The edge computing module connects to the communication relay module via 4G / 5G or satellite communication links, uploading the compressed and feature-extracted state data to the shore-based situation fusion center. The shore-based situation fusion center is equipped with a deep temporal neural network model, which is used to generate motion situation prediction results within a predetermined future time window based on historical state sequences and current multidimensional inputs.

[0039] In specific applications of this invention, after the system is started, multi-source sensor modules deployed at key locations on the unmanned surface vessel (USV) synchronously collect environmental and platform status information: the inertial measurement unit (IMU) outputs triaxial acceleration and angular velocity data at a frequency of not less than 200Hz, and the Global Navigation Satellite System (GNSS) receiver synchronously provides high-precision position, velocity, and time information; lidar and millimeter-wave radar actively detect surface and air targets respectively, generating point clouds and target tracks; the hydrophone array continuously monitors underwater acoustic signals to identify potential underwater obstacles or the activities of other vessels; and simultaneously, environmental parameter sensors acquire real-time wind speed, air pressure, seawater temperature and salinity, and wave spectrum marine meteorological elements.

[0040] As the shipboard data hub, the embedded main control unit first performs timestamp alignment and format standardization on the data streams from different sensors. In particular, a tight coupling mechanism is adopted for IMU and GNSS, sharing a high-stability crystal oscillator clock source at the hardware level to ensure microsecond-level synchronization accuracy. Subsequently, the main control unit pushes the packaged multi-dimensional raw data to the edge computing module in real time through the PCIeGen2x4 high-speed serial interface. The edge computing module adopts a heterogeneous computing architecture. Its ARM processor is responsible for executing the lightweight operating system task scheduling and data compression process (such as LZ4 or Zstandard algorithm), while the neural network acceleration unit (NPU) loads a quantized optimized convolutional-long short-term memory hybrid model to perform local feature extraction on the input data stream, such as identifying key state indicators of acceleration abrupt changes, heading angle inflection points, and obstacle approach trends. Thus, preliminary anomaly detection and situation assessment can be completed without relying on shore-based systems.

[0041] The compressed and feature-refined status data packets are handed over to the communication relay module by the edge computing module. The module integrates dual-mode communication capabilities, and under normal circumstances, it prioritizes the use of 4G / 5G cellular networks for low-latency backhaul. Once the signal strength is detected to be below -110dBm or the continuous packet loss rate exceeds 15%, it automatically and seamlessly switches to the L-band satellite communication link, and embeds the link identifier and redundant timestamp verification field in the data packet header to ensure that the shore-based end can accurately identify and reassemble the data stream transmitted through multiple paths. All uploaded data eventually converges to the shore-based situation fusion center, which is deployed on a high-performance cloud server cluster. The center uses a message queue mechanism to buffer, deduplicatize, and distribute data streams from multiple unmanned surface vessels.

[0042] The core of the shore-based situation fusion center is a deep temporal neural network model, which adopts an encoder and decoder structure: the encoder consists of three layers of bidirectional gated recurrent units (GRUs) to deeply mine long-term spatiotemporal dependencies in historical trajectories; the decoder combines an attention mechanism with two layers of long short-term memory (LSTM) units to dynamically fuse current wind speed, ocean current vectors, and external context information of the distribution of nearby targets when generating position, velocity, and heading prediction sequences updated at a frequency of 1Hz within the next 30 seconds. During the training phase, the model not only uses real navigation logs but also introduces enhanced samples based on high-fidelity ocean dynamics simulations and improves robustness to sensor anomalies or data loss scenarios through adversarial disturbance injection.

[0043] In one specific implementation, the inertial measurement unit (IMU) and the global navigation satellite system (GNSS) receiver are tightly coupled and integrated into the same package. The original IMU data is synchronized and timestamped with the GNSS pseudorange and carrier phase observations through an onboard FPGA chip. The IMU sampling frequency is no less than 200 Hz, and the GNSS receiver output frequency is no less than 10 Hz. The lidar and millimeter-wave radar are respectively mounted on the top of the bow mast and the side support of the unmanned surface vessel (USV). Their fields of view overlap by at least 30 degrees in the horizontal plane. The lidar output point cloud data frame and the millimeter-wave radar target track list are coordinate unified through a spatiotemporal registration module in the embedded main control unit. The module performs dynamic rotation and translation transformations on the radar's original coordinate system based on the USV's current roll, pitch, and heading angles.

[0044] In specific applications, during system operation, the Inertial Measurement Unit (IMU) and the Global Navigation Satellite System (GNSS) receiver are physically integrated into the same sealed enclosure. This tightly coupled structure not only reduces system errors caused by installation deviations but also creates the hardware prerequisite for deep data fusion between the two. The onboard FPGA chip acts as the underlying synchronization controller, aligning the original triaxial acceleration and angular velocity signals output by the IMU (sampling frequency not less than 200Hz) and the pseudorange and carrier phase observations provided by the GNSS receiver (update frequency not less than 10Hz) with hardware-level precision. The FPGA is equipped with a high-stability temperature-compensated crystal oscillator as a unified clock source, ensuring that the data from the two heterogeneous sensors are synchronized on a microsecond-level timescale. Through preset interpolation and extrapolation algorithms, during brief interruptions of the GNSS signal (such as entering under a bridge, in a canyon, or under electromagnetic interference), the IMU's high-frequency data maintains continuous and smooth position and attitude calculations, thereby improving the robustness and continuity of the unmanned surface vessel's positioning in complex sea conditions or obscured environments.

[0045] Meanwhile, the system's front-end perception layer constructs complementary surface and near-air target detection capabilities through lidar and millimeter-wave radar. The lidar is deployed at the top of the mast at the bow of the unmanned surface vessel, providing high-resolution 3D point clouds suitable for static obstacle identification and terrain mapping; the millimeter-wave radar is installed on the side bracket, possessing all-weather, rain and fog resistant, and light-obstruction penetration capabilities, effectively detecting moving targets and outputting a track list with speed information. The two radars have an overlap of no less than 30 degrees in the horizontal field of view, providing a spatial intersection basis for subsequent multi-source perception fusion. When the two radars simultaneously observe the same target, the spatiotemporal registration module in the embedded main control unit initiates the coordinate unification process: First, it obtains the current six-degree-of-freedom motion state of the hull from the IMU / GNSS fusion module, including the roll, pitch, and yaw angles; then, based on these real-time attitude parameters, it applies dynamic rotation matrix and translation vector transformations to the original point cloud coordinate system of the lidar and the target coordinate system of the millimeter-wave radar, respectively, and maps the two to the geographic north-east-sky (ENU) coordinate system with the center of gravity of the unmanned surface vessel as the origin.

[0046] Within this unified coordinate framework, the system further performs feature-level or target-level fusion: for example, it associates and matches target points with radial velocity detected by millimeter-wave radar with clusters of corresponding spatial regions in lidar point clouds, thereby giving point cloud targets that originally had no velocity information motion vectors. This effectively overcomes the performance degradation problem of single sensors in adverse weather, low visibility, or strong reflection interference scenarios, enabling the system to maintain stable and accurate perception of the surrounding situation in complex marine environments such as day and night, rain, fog, and splash.

[0047] In one specific implementation, the hydrophone array consists of four omnidirectional hydrophones arranged in a cross-shaped topology, positioned below the keel of the unmanned surface vessel at a depth of 0.8-1.2 meters above the water surface. The analog signals output by each hydrophone are pre-amplified by a low-noise amplifier and then connected to a multi-channel synchronous analog-to-digital converter with a sampling rate of not less than 96kHz. The signals are then transmitted to the embedded main control unit via an SPI bus. The embedded main control unit has a built-in direction-of-arrival (DOA) estimation algorithm module that uses the phase difference of the signals received by the four-element array to construct a covariance matrix, thereby calculating the azimuth and pitch angles of the underwater sound source. The edge computing module adopts a heterogeneous computing architecture, including an ARM Cortex-A72 multi-core processor and an NPU neural network acceleration unit. The ARM processor runs a lightweight Linux system and deploys a data compression algorithm to perform lossless compression based on LZ4 or Zstandard on the raw data from the multi-source sensors.

[0048] In specific applications of this invention, during system operation, a cross-shaped hydrophone array deployed below the keel of the unmanned surface vessel at a depth of 0.8 to 1.2 meters below the water surface continuously receives acoustic signals from the underwater space. The array consists of four omnidirectional hydrophones arranged in an orthogonal symmetrical layout to ensure uniform sound wave response characteristics within a 360-degree range on the horizontal plane and to retain a certain pitch resolution capability in the vertical direction. The weak analog sound pressure signal output by each hydrophone is first impedance matched and signal gain adjusted by an independent pre-amplifier to suppress the introduced electronic noise to the maximum extent and improve the signal-to-noise ratio.

[0049] Subsequently, the four amplified analog signals are synchronously fed into a multi-channel high-precision analog-to-digital converter (ADC). The converter performs time-aligned digitization processing on all channels at a sampling rate of no less than 96kHz to ensure that the phase relationship between the channels is not distorted due to sampling timing deviation. The high sampling rate not only meets the Nyquist criterion for complete capture of typical underwater sound source frequencies (usually concentrated in hundreds of hertz to tens of kilohertz), but also provides sufficient time-domain information redundancy for subsequent high-resolution direction of arrival (DOA) estimation. The digitized four-channel audio stream is transmitted to the embedded main control unit in real time via a high-speed SPI bus, where the initial acoustic signal processing task is completed.

[0050] The embedded main control unit integrates a dedicated direction-of-arrival (DOA) estimation algorithm module. Based on classic subspace methods (such as MUSIC or ESPRIT) or covariance matrix eigenvalue decomposition techniques, the module utilizes the phase difference caused by the propagation path differences of the same sound source signal received by the four-element array between different hydrophones to construct the cross-correlation matrix or covariance matrix of the multi-channel signal. By performing eigenvalue decomposition on the matrix, the signal subspace and noise subspace are separated, and then the spatial spectrum peak is searched. Finally, the azimuth and elevation angles of the underwater sound source relative to the UAV coordinate system are calculated. The entire process is completed on the UAV itself, without relying on shore-based computing resources. This allows the system to immediately generate spatial positioning information after detecting suspicious underwater targets (such as propeller noise from other vessels, underwater vehicles, marine biological activity, or man-made sound sources) and incorporate it into the overall state assessment as part of the environmental situation.

[0051] Meanwhile, to address the pressure on communication bandwidth caused by the massive amounts of raw data generated by multiple sensors (including the aforementioned hydrophone array, IMU, GNSS, and LiDAR), the system introduces an edge computing module as a local intelligent preprocessing hub. This module employs a heterogeneous computing architecture, with a high-performance ARM Cortex-A72 multi-core processor and a dedicated neural network processing unit (NPU) working in tandem. The ARM processor runs a lightweight Linux operating system, responsible for task scheduling, peripheral management, and general data processing. Specifically, it deploys efficient lossless compression algorithms (such as LZ4 or Zstandard) to compress raw or preprocessed data streams from various sensors in real time. These algorithms reduce the data volume required for subsequent wireless transmission while ensuring data integrity. For example, 96kHz sampled four-channel audio raw data can have its bandwidth usage reduced by more than 50% after Zstandard compression, and point clouds or IMU sequences can achieve similar compression ratios through structured coding.

[0052] In one specific implementation, the communication relay module includes a dual-mode communication unit, which integrates a 4G / 5G cellular communication chip and an L-band satellite modem, respectively. The two are dynamically switched by a link selection controller in the embedded main control unit. The shore-based situation fusion center is deployed on a cloud server cluster. Its input interface receives compressed status data streams from multiple unmanned surface vessels and buffers and distributes them through a message queue mechanism. The deep temporal neural network model adopts an encoder and decoder structure. The encoder is composed of three layers of bidirectional GRU units stacked together to extract long-term dependencies of historical trajectories. The decoder is composed of two layers of attention-enhanced LSTM units, which combine the current environmental wind speed, ocean current vector and the position of nearby targets as external context inputs, and output a predicted sequence of position, velocity and heading updated at a frequency of 1Hz within the next 30 seconds.

[0053] In practical applications of this invention, during system operation, the onboard communication relay module continuously monitors the wireless communication environment of the current area. Its integrated 4G / 5G cellular communication chip and L-band satellite modem form a dual-mode redundancy architecture, dynamically managed by a link selection controller in the embedded main control unit. The controller polls cellular signal strength (RSRP), signal-to-noise ratio (SINR), and continuous packet loss rate indicators at millisecond intervals. Once the cellular link quality is detected to degrade below a preset threshold (e.g., signal strength below -110dBm or packet loss rate exceeding 15%), a seamless handover mechanism is immediately triggered, automatically migrating the data transmission channel to the satellite link. During the handover process, the system embeds a unified timestamp, link identifier, and sequence number in the header of each data packet and employs forward error correction coding to enhance error resistance, ensuring that even with frequent link switching or partial data packet loss, the shore-based end can accurately identify, deduplicate, and sequentially reassemble the status data stream from the same unmanned surface vessel, thereby maintaining communication continuity and integrity.

[0054] Multidimensional state data (including position, attitude, velocity, obstacle distribution, underwater sound source location, and environmental parameters), compressed and feature-extracted by the edge computing module, is stably uploaded to the shore-based situation fusion center via the aforementioned adaptive communication link. The center is deployed on a highly available cloud server cluster, possessing elastic scaling and load balancing capabilities. Its front-end input interface receives asynchronous data streams from dozens or even hundreds of unmanned surface vessels (USVs) in a high-concurrency manner, first writing them into a distributed message queue (such as Kafka or RabbitMQ). The message queue not only serves to smooth out traffic spikes and buffer loads but also isolates and distributes data streams from different USVs to backend processing nodes through a topic partitioning mechanism, avoiding processing bottlenecks while ensuring data order and reliability. Subsequently, each processing node consumes data from the queue, performing time alignment, coordinate normalization, and outlier filtering standardization preprocessing to form a structured spatiotemporal state sequence, which serves as input to the deep prediction model. The core deep temporal neural network model adopts an encoder and decoder architecture. Due to the highly nonlinear, strongly coupled and sensitive to external disturbances of unmanned surface vessel motion in the marine environment, the encoder part is composed of three layers of bidirectional gated recurrent units (GRUs) stacked together. It can simultaneously scan historical trajectory sequences (usually containing state data from the past 60-120 seconds) from both forward and backward directions, fully capturing the long-term dynamic dependence characteristics of velocity change trends, heading inertia, and acceleration abrupt changes.

[0055] The decoder, based on context vectors and combined with real-time external environmental context information, including current wind speed vectors, ocean current direction and speed provided by meteorological and oceanographic databases, and the positions and motion states of nearby targets gathered from a multi-vessel collaborative sensing network, generates a refined situational prediction for the next 30 seconds. The decoder employs two layers of Long Short-Term Memory (LSTM) units and introduces an attention mechanism: at each prediction step (output at a 1Hz frequency), the attention module dynamically calculates the importance weights of each time step in the historical state, enabling the model to focus on the past segments that have the greatest impact on the current prediction (e.g., sharp turns or obstacle avoidance maneuvers within the last 5 seconds). Finally, the decoder outputs the position (latitude and longitude), velocity (magnitude and direction), and heading angle of the future trajectory frame by frame, forming a physically plausible and environmentally adaptable prediction path.

[0056] In one specific implementation, the training dataset of the deep temporal neural network model includes real navigation logs and simulation-enhanced samples. The simulation samples are generated by a high-fidelity ocean dynamics model, covering different sea state levels, wind angles, and obstacle densities. An adversarial perturbation mechanism is introduced during training, injecting random noise conforming to a Gaussian process distribution into the input sequence. The embedded main control unit and the edge computing module are interconnected using a PCIeGen2x4 high-speed interface, with a physical layer transmission rate of no less than 2GB / s, and a double-buffered circular queue mechanism is established between them.

[0057] In practical applications, before actual operation, the system first constructs a large-scale, multi-dimensional training dataset by integrating real-world data with a high-fidelity simulation environment. The real navigation logs are derived from complete state sequences collected by multiple unmanned surface vessels in different sea areas, seasons, and mission types during historical missions, containing high-dimensional temporal information on position, speed, attitude, environmental parameters, and operational commands. The simulation-enhanced samples are generated by a high-fidelity ocean dynamics model. This model simulates sea states from level 0 (calm) to level 9 (stormy), 360-degree omnidirectional wind direction changes, and obstacle distribution scenarios from sparse to dense. It can reproduce extreme conditions in the real ocean that are difficult to reproduce frequently but are extremely risky, such as severe rolling caused by cross waves, track deviation under strong crosswinds, or emergency collision avoidance in dense fishing boat groups.

[0058] To enhance the robustness of the model in real-world deployments when faced with sensor noise, data loss, or partially malfunctioning non-ideal inputs, an adversarial perturbation mechanism is introduced during training. In each training batch's input sequence, random noise conforming to a Gaussian process distribution is injected with a preset probability. Its covariance structure is customized based on the physical characteristics of each sensor (e.g., IMU drift characteristics, GNSS multipath error spectrum, and the sparsity of radar point clouds). The perturbation is not simple white noise, but rather structured interference with spatiotemporal correlation, simulating attitude jitter caused by wave turbulence, missing lidar point clouds due to rain and fog, or transient communication interruptions caused by electromagnetic interference in real-world environments. After repeated exposure to such "stress tests," the model learns to ignore irrelevant perturbations and focus on key dynamic features, thus enabling it to output stable and reasonable situational prediction results even when facing partially distorted inputs in actual operation.

[0059] Once the model training is complete and validated, it is quantized, pruned, and deployed to the onboard edge computing module. At this point, the efficient data path between the embedded main control unit and the edge computing module becomes crucial for ensuring real-time intelligent inference. The two are directly connected via a PCIeGen2x4 high-speed interface, with the physical layer providing a bidirectional transmission bandwidth of no less than 2GB / s, far exceeding the aggregation rate of raw data streams from multiple sensors (typically in the hundreds of MB / s range), fundamentally eliminating data bottlenecks caused by traditional serial interfaces (such as UART or USB). Based on this high-speed channel, the system further establishes a dual-buffered circular queue mechanism: the main control unit writes the packaged sensor data to the current write buffer, while the edge computing module concurrently reads and processes it from another read buffer; once full, it immediately switches to the backup buffer, achieving zero-copy, non-blocking data pipeline transmission. This mechanism not only ensures the synchronous delivery of heterogeneous data streams from high-frequency IMUs (above 200Hz) and high-bandwidth LiDAR point clouds, but also supports temporary buffering to prevent data loss during sudden increases in edge computing load (such as when sudden obstacle detection triggers complex inference).

[0060] Furthermore, since sensor noise and data incompleteness have been fully considered during the training phase, the edge computing module can still make reasonable inferences based on currently available data even under conditions of limited communication or local sensor anomalies. The high-speed interface ensures that all available information is sent to the NPU acceleration unit with minimal latency, enabling the lightweight neural network model to complete feature extraction and preliminary situation assessment within milliseconds. For example, when the hydrophone array detects an abnormal sound source, the lidar identifies a floating object ahead, and the IMU records increased ship roll, the multimodal signals arrive at the edge computing module almost simultaneously, triggering joint anomaly detection and rapidly uploading compressed key features to the shore-based system to update the global prediction model.

[0061] In one specific implementation, the environmental parameter sensors include a three-axis ultrasonic anemometer, a digital barometer, a seawater temperature / salinity probe, and a six-degree-of-freedom wave spectrometer, which are respectively installed on the top of the superstructure of the unmanned surface vessel, in the middle of the deck, on the hull outside the hull, and below the waterline of the bilge. The output signals of each sensor are connected to the dedicated ADC channel of the embedded main control unit after being isolated and amplified by a circuit. The sampling period is uniformly triggered by the internal timer of the main control unit.

[0062] In specific applications of this invention, during system operation, environmental sensors deployed at different physical locations on the hull work collaboratively to form a three-dimensional marine meteorological sensing network covering the atmosphere, sea surface, and underwater near-field. A three-axis ultrasonic anemometer located on the top of the superstructure continuously measures the three-dimensional wind vector (including wind speed, horizontal wind direction, and vertical wind component). Its absence of mechanically rotating parts avoids the jamming or corrosion problems of traditional wind cups in high-humidity and high-salt environments, ensuring long-term stable operation. A digital barometer installed in the middle of the deck monitors atmospheric pressure in real time with millibar-level accuracy. These changes provide crucial information for judging short-term weather trends (such as the approach of a low-pressure system); the seawater temperature / salinity probes fixedly installed on the hull directly contact the flowing seawater and continuously acquire the thermodynamic state of the surface seawater through the conductivity-temperature-depth (CTD) principle, participating in the local correction of the ocean current density-driven model; while the six-degree-of-freedom wave spectrometer deployed in the area below the waterline of the bilge uses inertial sensing and spectrum analysis technology to calculate the main direction, significant wave height, peak period and energy distribution of waves in real time, characterizing the excitation effect of the current sea state on the hull motion.

[0063] Before entering the embedded main control unit, the output signals of all the aforementioned sensors are first processed by a dedicated isolation amplification circuit. This circuit employs opto- or magnetic coupling isolation technology to effectively block common-mode noise introduced by potential differences in the hull's metal structure, surge currents, or electromagnetic interference. Simultaneously, it performs low-noise pre-amplification on weak analog signals (such as thermistor voltage and piezoelectric wave-sensing signals) to ensure the signal-to-noise ratio meets the requirements of high-precision analog-to-digital conversion. Subsequently, each signal is connected to a dedicated high-resolution ADC channel configured within the embedded main control unit. These channels have independent sample-and-hold circuits and programmable gain control, enabling them to adapt to the output amplitude range of different types of sensors.

[0064] Furthermore, the sampling actions of all ADC channels are not triggered independently, but are uniformly synchronized by a high-precision hardware timer inside the embedded main control unit. This ensures that the heterogeneous environmental parameters such as wind speed, air pressure, temperature and salinity, and waves are collected under a strictly consistent time reference, with the time deviation controlled within ±2 milliseconds. The hard synchronization mechanism completely eliminates the problem of physical field reconstruction distortion caused by sampling timing misalignment. For example, in complex sea conditions with strong gusts and swells, if there is a deviation of hundreds of milliseconds between wind speed and wave data, the wind-induced wave generation process will be incorrectly associated, thus misleading the motion model.

[0065] The collected synchronous environmental data is then integrated into the global state vector of the unmanned surface vessel (USV) and injected as contextual information into subsequent processing. On one hand, these parameters are directly used in the local dynamic compensation algorithm of the onboard edge computing module, such as correcting the wind-induced drift term in GNSS / IMU fusion positioning based on real-time wind speed vectors, or adjusting the attitude control PID parameters based on wave spectrum characteristics. On the other hand, after compression, the data is uploaded to the shore-based situation fusion center, becoming a key external input for the deep temporal neural network decoding stage. When predicting the trajectory for the next 30 seconds, the model not only relies on historical tracks but also dynamically integrates the influence of the current wind direction angle on the hull's thrust, the modulation of the drag coefficient by seawater density, and whether the wave dominant frequency and the hull's inherent roll frequency are close to the resonance physical mechanism, thereby improving the physical rationality and environmental adaptability of the prediction results.

[0066] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0067] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. An intelligent unmanned surface vehicle real-time state monitoring and situation prediction system, characterized in that, The application relates to a multi-source sensor module, an embedded main control unit, an edge computing module, a communication relay module and a shore-based situation fusion center arranged on the hull of an unmanned ship. The multi-source sensor module comprises an inertial measurement unit, a global navigation satellite system receiver, a laser radar, a millimeter wave radar, an underwater acoustic array and an environmental parameter sensor, and each sensor is connected to the embedded main control unit through a CAN bus or an Ethernet interface. The embedded main control unit is connected to the edge computing module through a high-speed serial interface and is used for packaging and transmitting original sensor data to the edge computing module for local preprocessing. The edge computing module is connected to the communication relay module through a 4G / 5G or satellite communication link and uploads state data compressed and extracted to the shore-based situation fusion center. The shore-based situation fusion center is provided with a deep time sequence neural network model and is used for generating a motion situation prediction result in a future predetermined time window based on a historical state sequence and a current multi-dimensional input. The inertial measurement unit and the global navigation satellite system receiver are integrated in the same packaging shell in a close-coupled mode and realize synchronous alignment and timestamp calibration of original IMU data and GNSS pseudo-range and carrier phase observation values through a built-in FPGA chip, wherein the IMU sampling frequency is not lower than 200Hz, and the GNSS receiver output frequency is not lower than 10Hz. 2.The intelligent unmanned ship real-time state monitoring and situation prediction system according to claim 1, characterized in that, The laser radar and the millimeter wave radar are respectively installed on the top end of the bow mast and the side bracket of the unmanned ship, the field angles of the two radars have at least a 30-degree overlapping region in the horizontal plane, and the point cloud data frame of the laser radar and the target track list of the millimeter wave radar are unified in coordinates through a space-time registration module in the embedded main control unit, and the module implements dynamic rotation and translation transformation on the original radar coordinate system according to the current roll angle, pitch angle and heading angle of the unmanned ship. 3.The intelligent unmanned surface vehicle real-time state monitoring and situation prediction system of claim 1, wherein, The underwater acoustic array is composed of four omnidirectional hydrophones arranged in a cross-shaped topology and is arranged below the keel of the unmanned ship at a depth of 0.8-1.2 meters from the water surface, analog signals output by the hydrophones are connected to a multi-channel synchronous analog-digital converter through a preamplifier with low noise, the sampling rate is not lower than 96kHz, and the signals are transmitted to the embedded main control unit through an SPI bus; the embedded main control unit is internally provided with a direction of arrival estimation algorithm module, a covariance matrix is constructed by using the phase difference of the signals received by the four-element array, and then the azimuth and elevation angles of the underwater sound source are solved. 4.The intelligent unmanned ship real-time state monitoring and situation prediction system of claim 1, wherein, The edge computing module adopts a heterogeneous computing architecture and comprises an ARM Cortex-A72 multi-core processor and an NPU neural network acceleration unit, the ARM processor runs a lightweight Linux system and deploys a data compression algorithm, and lossless compression is performed on the original data of the multi-source sensor based on LZ4 or Zstandard.

5. The intelligent unmanned surface vehicle real-time state monitoring and situation prediction system according to claim 1, wherein, The communication relay module comprises a dual-mode communication unit, a 4G / 5G cellular communication chip and an L-band satellite modem are integrated in the dual-mode communication unit, and the two are dynamically switched through a link selection controller in the embedded main control unit. 6.The intelligent unmanned surface vehicle real-time state monitoring and situation prediction system of claim 1, wherein, ​ 7. The intelligent unmanned surface vehicle real-time state monitoring and situation prediction system according to claim 1, wherein, The shore-based situation fusion center is deployed in a cloud server cluster, an input interface of the shore-based situation fusion center receives compressed state data streams from multiple unmanned ships, and a message queue mechanism is used for buffering and distribution; the deep time sequence neural network model adopts an encoder and decoder structure, the encoder is composed of three layers of bidirectional GRU units stacked, is used for extracting long-term dependence of historical trajectories, the decoder is composed of two layers of attention enhanced LSTM units, combines current environmental wind speed, sea current vector and adjacent target position as external context input, and outputs a position, speed and heading prediction sequence updated at a frequency of 1 Hz within 30 seconds in the future. 8.The intelligent unmanned ship real-time state monitoring and situation prediction system of claim 1, wherein, The training data set of the deep time sequence neural network model includes real navigation logs and simulation enhanced samples, the simulation samples are generated by a high-fidelity ocean dynamics model, cover different sea state levels, wind direction angles and obstacle density scenes, and an adversarial disturbance mechanism is introduced in the training process to inject random noise conforming to a Gaussian process distribution in the input sequence. 9.The intelligent unmanned surface vehicle real-time state monitoring and situation prediction system of claim 1, wherein, The embedded host unit and the edge computing module are interconnected by using a PCIe Gen2x4 high-speed interface, the physical layer transmission rate is not less than 2 GB / s, and a double-buffer ring queue mechanism is established between the two.

10. The intelligent unmanned surface vehicle real-time state monitoring and situation prediction system according to claim 1, characterized in that, The environmental parameter sensors include a three-axis ultrasonic anemometer, a digital barometer, a seawater temperature / salinity probe and a six-degree-of-freedom wave spectrum instrument, which are respectively installed on the top of the superstructure, the middle of the deck, the outside of the hull bottom and the position below the waterline of the bilge of the unmanned ship; the output signals of the sensors are connected to the special ADC channels of the embedded host unit after isolation and amplification, and the sampling period is triggered by the internal timer of the host unit.

Images