LNGC shore data compression and efficient backhaul method

Abstract

The application discloses an LNGC ship-shore data compression and high-efficiency backhaul method, which proposes a set of end-to-end technical system including multi-source heterogeneous data collection, backhaul data classification and division, lossless compression of high-frequency time sequence data characteristics, dynamic protocol network transmission, intelligent network interruption and continuous transmission and the like, and can systematically solve a series of core problems such as large data volume, limited satellite bandwidth, unreliable network environment and heterogeneous data sources in the process of constructing an LNGC digital twin, thereby breaking through the technical bottleneck of data integrity and high-efficiency backhaul faced by the current LNG ship digital twin, and laying a solid methodological foundation for the creation and application of a high-fidelity digital twin.

Description

Technical Field

[0001] This invention relates to the field of ship intelligence and ocean communication data transmission technology, and in particular to a method for LNGC ship-to-shore data compression and efficient backhaul. Background Technology

[0002] Against the backdrop of a global energy transition towards cleaner and lower-carbon energy sources, liquefied natural gas (LNG), as a key transitional energy source, has seen a continuous increase in its maritime trade volume. LNG carriers (LNGCs), as core logistics carriers and high-value assets in the global LNG supply chain, are undergoing intelligent and digital transformation in their operation and management. Improving the operational safety, economic efficiency, and environmental friendliness of LNGCs has become a core issue for the shipping and energy sectors.

[0003] Under this trend, intelligent ship technology, especially digital twin technology, is regarded as a key enabling technology for achieving refined management of the entire life cycle of ships. Digital twins, by constructing a virtual model that is synchronously and faithfully mapped in real time to the physical ship, can achieve deep perception of the ship's status, simulation and deduction of the operational process, predictive maintenance for fault diagnosis, and global optimization of energy efficiency, thus providing powerful support for shore-based decision support. However, the "high fidelity" and practical value of digital twins fundamentally depend on the continuous, complete, and reliable acquisition of high-frequency, multi-dimensional operational data from the physical ship. This data covers temperature and pressure distribution throughout the cargo tanks, real-time BOG generation rate, instantaneous energy consumption and emission parameters of main and auxiliary engines, stress and strain of the hull structure, real-time trajectory, draft, and thousands of equipment status and alarm information, forming a vast, heterogeneous, and rapidly updated data system.

[0004] LNG carriers primarily navigate ocean and transoceanic routes, and their data communication with shore-based data centers relies almost entirely on maritime satellite communication links. This specific physical and engineering constraint presents the following three severe technical challenges for ship-to-shore data backhaul: (1) Challenges of massive data and limited transmission channels: To achieve high-fidelity mapping, the raw data collected by the dense shipborne sensor network is enormous, with the daily data volume of a single ship reaching tens of gigabytes. However, maritime satellite communication bandwidth is limited and the cost is high. The huge data volume and the narrow, expensive transmission channels constitute the primary bottleneck.

[0005] (2) Challenges of real-time business requirements and uncertain network environment: Business scenarios such as ship safety monitoring, fault early warning and critical equipment status monitoring require data to be transmitted with low latency and high reliability. However, satellite links suffer from high latency and high bit error rate, and signal quality is easily affected by severe sea conditions, weather conditions and geographical location, resulting in unstable network connection and seriously threatening the timely and complete transmission of critical data.

[0006] (3) The challenge of unifying multi-source heterogeneous data and system integration: The integrated automation system of LNGC consists of many subsystems from different manufacturers and following different industrial communication protocols (such as Modbus, Profibus, CAN, etc.), with different data formats and interface standards. Efficiently and standardizedly integrating such heterogeneous multi-source data and incorporating it into a unified and stable feedback channel is a complex systems engineering challenge.

[0007] It is worth noting that in the broader field of ship data backhaul, some related technologies have been explored. For example, patent CN115589493A discloses a satellite transmission data compression method for ship video backhaul. This method significantly improves the compression efficiency of surveillance video on satellite channels by using inter-frame differential and bit layering, and performing run-length statistics in three dimensions: row, column, and bit depth, combined with window rotation to maximize run-length. However, this scheme is a single video compression algorithm, which does not cover the time-series sensor data characteristics required for LNGC digital twins, and it is not coordinated with reliable transmission mechanisms, thus failing to guarantee the integrity of data transmission under satellite links.

[0008] Patent CN106846917A discloses a remote automatic switching system for ship AIS data backhaul and its application method. This method designs a data backhaul and control mechanism for the Automatic Identification System (AIS) based on the VHF band (AIS channel), remotely controlling the switching of shipborne equipment by detecting signal status from shore-based sources, aiming to save communication costs and expand coverage. However, this solution is only suitable for small-volume, low-frequency AIS information, which is orders of magnitude smaller than the massive high-frequency industrial data required for LNGC digital twins in terms of both volume and real-time performance. Furthermore, it relies on specific shore-based AIS base station infrastructure, making it unsuitable for open ocean areas lacking such coverage.

[0009] Patent CN120151019A discloses a data backhaul method based on new energy-powered ships. This method proposes a complete process for lightweight packaging of real-time ship operation data, defining verification methods, setting up encrypted transmission mechanisms, and issuing ship-side commands. Its contribution lies in considering the entire data process from acquisition to application and emphasizing data security. However, the "lightweight processing" is rather general, failing to provide specific algorithms for efficient and lossless compression tailored to the multi-source heterogeneous data characteristics of LNGCs. Furthermore, the solution does not fully consider the inherent instability and long-term interruption scenarios of satellite links, lacking matching reliability enhancement mechanisms such as intelligent caching and breakpoint resumption to ensure 100% data integrity.

[0010] As can be seen from the aforementioned patents, existing technical solutions are mostly limited to local optimization of the processing or transmission process of specific types of data. They have inherent defects such as narrow applicable scenarios, insufficient coordination among various links, and heavy reliance on specific infrastructure. They have failed to systematically solve the problems of massive data and limited bandwidth, real-time services and unreliable networks, and heterogeneous data and unified backhaul in the LNGC digital twin scenario. Summary of the Invention

[0011] In view of this, the present invention provides a method for LNGC ship-to-shore data compression and efficient backhaul. This invention integrates key functions and steps such as multi-source heterogeneous data acquisition, backhaul data classification and partitioning, lossless compression of high-frequency time-series data features, dynamic protocol network transmission, and intelligent network interruption resumption. On the one hand, data is partitioned based on backhaul classification rules and dynamically transmitted via network based on data transmission mechanisms, achieving value extraction and load optimization at the data source, prioritizing the real-time accuracy of critical safety and status data. On the other hand, it employs the GZip lossless compression algorithm optimized for the characteristics of high-frequency time-series ship data, and integrates the MQTT reliable transmission protocol and intelligent network interruption resumption mechanism. While ensuring 100% complete data backhaul, it significantly reduces dependence on satellite bandwidth, forming a closed-loop technical system of "data acquisition - value extraction - efficient compression - reliable transmission." This achieves efficient, complete, and reliable real-time backhaul of massive amounts of data from the LNGC digital twin under limited, expensive, and unstable satellite communication conditions.

[0012] A method for LNGC ship-to-shore data compression and efficient backhaul includes the following steps: S1: Collect multi-source heterogeneous raw ship operation data from the LNG ship's own IAS (Integrated Automation System) system and preprocess the raw data; S2: Based on the set data feedback classification rules, the preprocessed raw ship operation data is divided into Class A real-time data and Class C terminal data; S3: Determine the dynamic attributes of the real-time data of type A. If the real-time data of type A is trigger-type data and / or polling-type data that needs to be returned in real time, then compress the real-time data of type A efficiently according to the transmission mechanism and return it to the shore in a timely manner through the current network. If the real-time data of type A is other polling data, then determine whether the current communication network is a 5G network. If the current communication network is a 5G network, then compress the data efficiently and transmit it back to the shore in a timely manner through the current communication network. If the current communication network is a satellite network, then first compress the data efficiently, then downgrade the compressed data packet and mark it as type B historical data storage on the ship, and wait for the communication network to switch to a 5G network before transmitting this type of data back to the shore. S4: Store Class C terminal data directly on the ship's local machine, and then transmit it back to the shore in batches after the ship docks.

[0013] Preferably, in step S1, when preprocessing the raw ship operation data, a time series processing method based on a sliding window is first used to filter out noise and perform deduplication, and then the statistical values ​​of the data are calculated based on a set time interval (sampling frequency).

[0014] Preferably, in step S2, when the preprocessed original ship operation data is divided into Class A real-time data and Class C terminal data according to the set data feedback classification rules, Class A real-time data simultaneously covers three important dynamic data: equipment status, cargo hold safety monitoring, and safety alarms, and adopts a transmission trigger strategy of reporting when there is a change. Class C terminal data includes general static data such as equipment operation logs, historical alarm records, and conventional process parameters.

[0015] Preferably, in step S3, if data transmission fails or the current communication network is interrupted during the process of transmitting Class A real-time data back to the shore, the compressed data packet is downgraded and marked as Class B historical data packet and stored locally on the ship. After the communication network is restored, the stored Class B historical data packet is transmitted back to the shore first.

[0016] Preferably, in step S3, the GZip lossless compression algorithm is used to efficiently compress the data.

[0017] Preferably, in step S3, the QoS 1 or QoS 2 quality of service level is configured for the data packets of Class A real-time data and Class B historical data data packets through the MQTT protocol, and the compressed data packets are published to the communication network.

[0018] Preferably, in step S3, after the ship docks, the Class C terminal data is transmitted to the shore in batches via the port wireless network.

[0019] A ship-to-shore data compression and efficient backhaul system based on the above-described method includes a ship-side data processing system deployed on a ship and a shore-side data processing system deployed on shore. The ship-side data processing system is used to collect multi-source heterogeneous raw ship operation data from the IAS system of LNG ships and preprocess the collected data. Then, according to the set data return classification rules, the preprocessed raw ship operation data is divided into Class A real-time data and Class C terminal data. Based on the dynamic attributes and transmission mechanism of Class A real-time data, its dynamic network transmission strategy is determined for return to the shore. Class C terminal data is stored directly on the ship and returned to the shore in batches after the ship docks. Class A data packets that fail to be transmitted in real time are downgraded and marked as Class B historical data packets and stored on the ship for retransmission. The shore-based data processing system is used to subscribe to, receive, and decompress compressed data sent by the ship-based data processing system. It pushes the decompressed data to the shore-based digital twin application platform in real time, and simultaneously stores all decompressed data in the shore-based database.

[0020] Preferably, the shipboard data processing system includes a data acquisition module, a data processing module, a data compression module, and a data transmission and security module. The data acquisition module is used to collect multi-source heterogeneous raw ship operation data from the IAS system of LNG ships; The data processing module preprocesses the raw ship operation data. First, it categorizes the preprocessed raw data into Class A real-time data and Class C dock data according to predefined data return classification rules. Then, based on the attributes of Class A real-time data and the dynamic transmission network, it selects an appropriate transmission strategy, i.e., whether to store some data as Class B historical data on the ship. Finally, according to the dynamic network transmission strategy, it transmits Class A real-time data and Class B historical data to the data transmission and support module, while storing Class C dock data locally on the ship. In the event of data transmission failure or a detected communication network interruption, the compressed data packets that were not successfully transmitted in real-time are downgraded and marked as Class B historical data packets and stored locally on the ship. Once the communication network is restored, these data packets are prioritized for transmission back to the shore. The data compression module is used to compress the Class A real-time data that needs to be transmitted back to the shore using the GZip lossless compression algorithm based on the DEFLATE format, and then transmit the compressed package to the data transmission and protection module. The data transmission and security module is used to transmit data packets of Class A real-time data and Class B historical data back to the shore in a timely manner through the corresponding communication network based on the MQTT protocol's service classification strategy.

[0021] Preferably, the shore-based data processing system includes a data receiving and decompression module and a data distribution and application module. The data receiving and decompression module is used to subscribe to and receive compressed data sent by the ship's data processing system, reply with confirmation messages according to the MQTT protocol specification, and decompress the successfully received data using GZip. The data distribution and application module is used to push the decompressed data to the onshore digital twin application platform in real time for 3D visualization and alarm analysis, and to synchronously store all decompressed data in the onshore database.

[0022] The beneficial effects of this invention are: 1. This invention classifies raw ship operation data into Class A real-time data and Class C terminal data based on data backhaul classification rules. Then, it performs dynamic network transmission according to the transmission mechanism of Class A real-time data, effectively reducing the invalid transmission of non-critical data from the source. It effectively solves the existing contradictions between massive data and limited transmission channels, real-time services and unreliable networks, and heterogeneous data and unified backhaul in the LNGC digital twin scenario. Without sacrificing data integrity, it achieves efficient compression and reliable transmission of massive ship-shore data, ensures the real-time backhaul of critical safety and status data, and minimizes total bandwidth consumption while maximizing the integrity of data transmission.

[0023] 2. This invention proposes an end-to-end technical system that includes functions such as multi-source heterogeneous data acquisition, classification and division of backhauled data, lossless compression of high-frequency time-series data features, dynamic protocol network transmission, and intelligent network interruption resumption. It can systematically solve a series of core problems in the construction of LNGC digital twins, such as large data volume, limited satellite bandwidth, unreliable network environment, and heterogeneous data sources. This breaks through the current technical bottlenecks in data integrity and backhaul efficiency faced by LNG ship digital twins, and lays a solid methodological foundation for the creation and application of high-fidelity digital twins.

[0024] 3. This invention employs the GZip lossless compression algorithm, achieving a highly efficient and stable compression ratio that significantly outperforms other traditional compression algorithms. While ensuring data integrity, it reduces satellite transmission load by an order of magnitude, effectively alleviating transmission pressure under limited bandwidth. Furthermore, through the reliable transmission strategy of the MQTT reliable transmission protocol and intelligent network interruption resumption mechanism, it significantly reduces dependence on satellite bandwidth while ensuring 100% complete data transmission. This forms a closed-loop technical system of "data acquisition - value extraction - efficient compression - reliable transmission," systematically overcoming the core technical obstacles to efficient, complete, and reliable transmission of massive amounts of high-fidelity ship operation data under limited satellite bandwidth conditions. This lays a solid technical foundation for its engineering application in building high-fidelity LNGC digital twins.

[0025] 4. The physical entity system of the present invention can achieve efficient compression to reduce satellite communication traffic, significantly reduce operating costs, and ensure that the data transmitted back to the digital twin platform has high integrity and availability, meeting the stringent requirements of high-fidelity mapping and real-time analysis. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is the overall architecture diagram of the LNGC ship-to-shore data compression and high-efficiency backhaul system.

[0028] Figure 2 This is a schematic diagram illustrating the implementation principle of the GZip compression algorithm.

[0029] Figure 3 This is a flowchart of data transmission from the ship to the shore. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is described below with reference to specific embodiments shown in the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0031] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0032] To better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings.

[0033] This invention proposes a method for LNGC ship-to-shore data compression and efficient backhaul, specifically including the following steps: S1: Collect multi-source heterogeneous raw ship operation data from the LNG ship's own IAS (Integrated Automation System) and preprocess the raw data; S2: Based on the set data feedback classification rules, the preprocessed raw ship operation data is divided into Class A real-time data and Class C terminal data; S3: Determine the dynamic attributes of the real-time data of type A. If the real-time data of type A is trigger-type data and / or polling-type data that needs to be returned in real time, then compress the real-time data efficiently according to the transmission mechanism and return it to the shore in a timely manner through the current network. If the real-time data of type A is other polling data, then determine whether the current communication network is a 5G / 4G network. If the current communication network is a 5G / 4G network, then compress the data efficiently and transmit it back to the shore in a timely manner through the current communication network. If the current communication network is a satellite network, then first compress the data efficiently, then downgrade the compressed data packets and mark them as type B historical data stored locally on the ship, waiting for the communication network to switch to a 5G / 4G network before transmitting this type of data back to the shore. S4: Store Class C terminal data directly on the ship's local storage, and then transmit it back to the shore in batches after the ship docks. The following example, a 174,000 cubic meter LNGC vessel, illustrates in detail the LNGC ship-to-shore data compression and efficient data transmission method of this invention.

[0034] like Figure 1 As shown, the LNGC is equipped with a ship-side data processing system and a shore-side data processing system. The ship-side data processing system and the shore-side data processing system are connected through a communication network.

[0035] The shipboard data processing system is responsible for collecting multi-source heterogeneous data from the ship's integrated automation system, preprocessing, classifying and compressing it, and then transmitting it back via satellite link; the shore-based data processing system is responsible for reliably receiving and decompressing the data, and driving the analysis of digital twins and historical data.

[0036] Specifically, based on multi-objective optimization of data integrity, real-time performance, and bandwidth efficiency, the LNGC ship-to-shore data compression and efficient backhaul method of the present invention includes the following steps: S1: Data Acquisition and Preprocessing First, the hardware and software environment of the ship's data processing system needs to be initialized and configured. An industrial server with at least an 8-core CPU, 16GB of RAM, and a 1TB NVMe solid-state drive should be deployed on board. A communication connection should be established with the ship's IAS system via the OPCUA protocol interface.

[0037] Then, the ship's data processing system is used to collect multi-source heterogeneous raw ship operation data from the ship's IAS system.

[0038] Raw data on ship operation includes, but is not limited to, cargo tank temperature and pressure, BOG generation rate, main engine and auxiliary engine power, fuel consumption rate, ship speed, draft, and alarm signals of various equipment. This data is collected through the OPC UA protocol interface.

[0039] As shown in Table 1 below, the system subscribes to monitor 5,000-6,000 key data points throughout the ship, covering categories such as equipment status, cargo hold data, and safety data. The collection strategy is initialized to "monitoring mode" and "report on change", that is, collection is triggered only when the value of a data point changes beyond a preset threshold (for example, collection is triggered when the value of a data point changes beyond 0.5% of its range), thereby reducing redundant data traffic.

[0040]

[0041] The raw ship operation data collected needs to be preprocessed, that is, firstly, a time series processing method based on a sliding window is used to filter out noise and perform deduplication, and then the statistical values ​​of the data are calculated based on the set time interval (sampling frequency).

[0042] In this embodiment, there are three data preprocessing methods: pass-through mode, global preprocessing mode, and special preprocessing mode. Pass-through mode refers to the mode of directly forwarding a small amount of data that does not require processing; global preprocessing mode refers to performing noise reduction (such as 3σ filtering) and deduplication operations on all data. For example, a time series method based on a sliding window is used to treat data points that exceed three times the standard deviation of the mean of the past 10 sample values ​​as anomalies and filter or smooth them out; special preprocessing mode refers to performing statistical calculations on specific data groups (such as equipment status, cargo hold data) based on a set time interval (sampling frequency). For example, it calculates the "average within 1 minute", "maximum and minimum values ​​within 3 minutes", or "standard deviation within 5 minutes".

[0043] The data preprocessing methods for various types of data in the original ship operation data can be set according to the shipowner's needs.

[0044] S2: According to the set data feedback classification rules, the preprocessed raw ship operation data is divided into Class A real-time data and Class C terminal data.

[0045] As shown in Table 1 above, the transmission priority and transmission method of each type of data need to be set in advance according to the data characteristics. Then, the original ship operation data can be divided into Class A real-time data and Class C terminal data according to the defined "priority" and "transmission method".

[0046] Category A real-time data: This corresponds to "high" priority data, which includes equipment status data related to ship safety and core equipment status, cargo hold safety monitoring data, safety alarm data, and main engine emergency shutdown signals. It adopts a real-time transmission triggering strategy that reports any changes.

[0047] Category C terminal data (non-real-time batch data): Corresponding to "medium" and "low" priority data, equipment operation logs, historical alarm records, and routine process parameters are classified as Category C terminal data. The "timed packaging and onshore transmission" strategy is adopted. For example, routine equipment operating status parameters (such as cooling water temperature and lubricating oil pressure) and periodic performance report data are classified as Category C terminal data, configured to be packaged once per hour or per day, and automatically triggered in batch transmission after the ship enters the port's mobile communication network coverage area.

[0048] Among them, real-time data of type A can be further divided into trigger-type data and polling-type data according to its data transmission mechanism.

[0049] S3: Determine the data transmission mechanism of Class A real-time data. If Class A real-time data is triggered data and / or polling data that needs to be returned in real time (this type of polling data is manually set according to the shipowner's needs), then the Class A real-time data will be efficiently compressed and returned to the shore in a timely manner through the current network. If the real-time data of type A is other polling data (polling data that does not require real-time back transmission), then determine whether the current communication network is a 5G / 4G network. If the current communication network is a 5G / 4G network, then the real-time data of type A is efficiently compressed and immediately transmitted back to the shore via the current communication network (regardless of whether the current communication network is a satellite network or a 5G / 4G network, the key data of type A is transmitted back to the shore). If the current communication network is a satellite network, then the real-time data of type A is efficiently compressed, and its compressed data packet is downgraded and marked as a historical data packet of type B and stored locally on the ship. When the current communication network is switched to a 5G / 4G network, this type of data packet is transmitted back to the shore. S4: Store Class C terminal data directly on the ship's local machine, and then transmit it back to the shore in batches after the ship docks.

[0050] For Class A critical data that needs to be transmitted back via satellite network or 5G / 4G network, the GZip lossless compression algorithm is used for data compression. The core of this compression algorithm lies in the use of the DEFLATE compression format, which combines the LZ77 algorithm with the Huffman coding algorithm to deeply eliminate data redundancy and achieve a high compression ratio, thereby improving the efficiency of massive data transmission. Its design process follows the principle of layered processing, and the specific design process is as follows: Figure 3 As shown: 1) A sliding window mechanism based on the LZ77 algorithm scans the input data (i.e., real-time data of type A), identifies and locates repeating string sequences, and replaces them with a simplified tuple (distance, length) to eliminate local redundancy and achieve dictionary-based initial compression. Here, "distance" represents the offset from the current position to the first occurrence of the string; "length" represents the length of the matched string.

[0051] 2) The Huffman coding algorithm is used to statistically analyze the output of the LZ77 stage, and the optimal prefix code (Huffman tree) is constructed based on the frequency of byte occurrence. Bytes with high occurrence frequency are assigned shorter codewords, while bytes with low frequency are assigned longer codewords, thereby minimizing the total number of bits after encoding and achieving further compression based on entropy coding.

[0052] 3) The implementation uses the DEFLATE algorithm for block partitioning. The input data is divided into multiple independent data blocks, and each block can dynamically or statically select the Huffman coding table based on the statistical characteristics of its internal data.

[0053] This modular design not only improves the algorithm's adaptability to various data characteristics but also allows for streaming processing, enhancing the feasibility and effectiveness of large-scale data compression. Compression performance is measured by data compression ratio and compression speed to determine LNGC data return efficiency, which can be modeled as follows: 1. Data compression ratio Data compression ratio is a core metric for evaluating the effectiveness of data compression algorithms. It is typically expressed as the ratio of the actual compressed data volume to the original data volume, denoted as ∑_(i=1)^∞ (2πf). A .

[0054]

[0055] In formula (1) The original data volume, The ratio represents the remaining data volume after compression. The difference between the two values ​​represents the actual compressed data volume after applying the compression algorithm. The larger the ratio, the better the compression effect.

[0056] 2. Data compression speed Data compression speed is another important performance indicator besides compression ratio. Assuming... This represents the original data volume before compression. The compression speed is the time required for compression. It can be represented as:

[0057] The compression implementation follows the three steps mentioned above, employing the GZip algorithm based on the DEFLATE format, combined with LZ77 sliding window compression and Huffman entropy coding to efficiently compress the data, achieving the optimal balance between compression speed and compression ratio for massive LNGC data. Real-ship data testing showed a compression ratio exceeding 93% for a single 1000KB data packet.

[0058] The compressed Class A real-time data is published to the communication network via the MQTT protocol. MQTT is an extremely lightweight publish / subscribe messaging protocol. The ship-side data processing system acts as the publisher, and the shore-side data processing system acts as the subscriber. Table 2 shows the three QoS levels defined by the MQTT protocol to ensure reliable transmission:

[0059] In this embodiment, a quality of service (QoS) level is configured (at least once) for Class A real-time data. Under this mechanism, after the ship publishes a message, it must receive a PUBACK acknowledgment packet from the shore to prove that the data transmission was successful; otherwise, it will time out and retransmit. Its state transition function can be modeled as follows:

[0060] For extremely critical alarm data, QoS 2 (Exactly Once) level can be configured: a four-way handshake: First, a QoS 2 command and a PUBLISH message containing a unique message identifier are sent to both the sender and receiver; second, upon successful reception, the receiver replies with a PUBREC message for confirmation, and the sender accordingly stops retransmitting the PUBLISH message and sends a PUBREL message to notify the receiver to prepare to release the message identifier; finally, the receiver replies with a PUBCOMP message to complete the entire transmission task, ensuring that the message is neither lost nor duplicated during transmission, achieving reliable "exactly once" delivery. The specific implementation is as follows:

[0061] When data is published to the communication network, its publication topic follows a hierarchical naming convention, such as lngc / {ship IMO number} / telemetry / {equipment area} / {data type}. The condition for determining transmission failure is: no PUBACK acknowledgment packet is received from the shore within a preset timeout period (set to 4-8 seconds based on the average delay of the satellite link).

[0062] For Class A real-time data that fails to be transmitted, its data packets are automatically downgraded and marked as Class B historical data packets, cached locally on the ship, and entered into the retransmission queue for re-upload.

[0063] To address communication network interruptions, the shipboard data processing system integrates an intelligent network failure recovery unit. This unit continuously monitors the network status and the reception status of MQTT protocol PUBACK acknowledgment packets. If a data packet does not receive an acknowledgment within the timeout period (T_timeout, set to 4-8 seconds depending on link latency), or if a network link interruption is directly detected, the compressed data packet and its metadata are immediately downgraded and marked as a Class B historical data packet and stored in a local circular buffer queue on the ship. Once the communication network is restored, the corresponding Class B historical data packet is prioritized for transmission back to the shore (i.e., data packets are automatically read sequentially from the head of the buffer queue and republished via the MQTT protocol until the queue is empty). This mechanism, in conjunction with the MQTT protocol's QoS mechanism, constitutes a double defense to ensure 100% complete data transmission.

[0064] The intelligent caching mechanism in this application is managed by an independent daemon process. This process monitors the network interface status and MQTT client connection status. Once a caching condition is triggered (data transmission failure, network failure, or interruption), the data packet and its metadata (including target topic, QoS level, and timestamp) are appended to a circular buffer file on the ship's solid-state drive in serialized format. This file has power-loss protection. The circular buffer has a capacity level (e.g., the buffer is 80% full). When the level is too high, the system can automatically trigger the cleanup or further compression of Class C terminal data to prioritize the caching space for Class B historical data packets.

[0065] Network recovery is determined based on dual checks of the satellite modem link status and the MQTT protocol layer connection status. After the resume transmission process begins, the cache management process reads data packets from the circular buffer file in a first-in-first-out order and republishes them through the restored MQTT connection. During the resume transmission, newly generated Class A real-time data is temporarily placed in a pending-send queue and sent only after the cache queue is cleared, thus ensuring that the integrity of historical data is prioritized for recovery.

[0066] In addition, for Class C terminal data, the data is transmitted to the shore in batches via the port's wireless network after the ships dock.

[0067] The present invention also provides a ship-to-shore data compression and efficient backhaul system based on the above method, including a ship-side data processing system deployed on a ship and a shore-side data processing system deployed on shore.

[0068] The shipboard data processing system includes a data acquisition module, a data processing module, a data compression module, and a data transmission and support module. The signal input of the data acquisition module establishes a communication connection with the ship's IAS system via the OPC UA protocol interface; its signal output is connected to the signal input of the data processing module; the signal output of the data processing module is connected to the signal input of the data compression module; and the signal output of the data compression module is connected to the data transmission and support module. Specifically: The data acquisition module is used to collect multi-source, heterogeneous raw ship operation data from the LNG ship IAS system.

[0069] The data processing module preprocesses the raw ship operation data. First, it categorizes the preprocessed raw data into Category A real-time data and Category C terminal data according to the established data return classification rules. Then, based on the attributes of Category A real-time data and the dynamic transmission network, it selects an appropriate transmission strategy, i.e., whether to store some data as Category B historical data on the ship. Finally, according to the dynamic network transmission strategy, it transmits Category A real-time data and Category B historical data to the data transmission and support module, while storing Category C terminal data locally on the ship.

[0070] The data compression module adds a header containing a timestamp, data category, and unique sequence number to the Class A real-time data that needs to be transmitted back to the shore. Then, it uses the GZip lossless compression algorithm based on the DEFLATE format to compress the data, generating a compressed packet, which is then transmitted to the data transmission and assurance module. The compressed data packet structure includes a header, a compressed payload, and a cyclic redundancy check (CRC) code.

[0071] The data transmission and assurance module is used to transmit Class A real-time data packets back to the shore via the 5G communication network in a timely manner based on the MQTT protocol and dynamic network transmission strategy. If the Class A real-time data transmission fails or the communication network fails, the Class A real-time data packets are downgraded and marked as Class B historical data packets and cached in the circular buffer of the ship's solid-state hard drive until the network is restored.

[0072] The shore-based data processing system includes a data receiving and decompression module and a data distribution and application module.

[0073] The data receiving and decompression module is used to subscribe to and receive compressed data sent by the ship's data processing system, reply with acknowledgment messages according to the MQTT protocol specification, and decompress successfully received data using GZip. Specifically, the MQTT broker cluster deployed on shore is responsible for receiving and temporarily storing messages; subscribing clients reply with acknowledgment as required after successfully receiving messages; the decompression service runs as an independent microservice, subscribing to topics from the broker cluster, obtaining compressed data packets, decompressing them using GZip, restoring the original data with headers, verifying their CRC checksums, and finally delivering the successfully decompressed data packets to the message middleware.

[0074] The decompressed data is distributed to applications based on its category identifier: (1) Class A real-time data: pushed to the digital twin application platform in real time to drive the 3D model update, virtual dashboard refresh and instant alarm.

[0075] (2) All Class B historical and Class C terminal data: synchronously stored in the shore-based time series database for historical data playback, trend analysis and decision support.

[0076] The data distribution and application module is used to push decompressed data to the onshore digital twin application platform in real time via WebSocket or a proprietary streaming data interface, and synchronously store all decompressed data in the onshore database. The digital twin application platform uses the received data to drive the status update of corresponding components of the 3D ship model, refresh the virtual instrument panel readings, and trigger visual alarms in 3D space when values ​​exceed limits.

[0077] The shore-based historical database is a time-series database that uses data tags to create indexes, supporting millisecond-level data writing and complex time-range queries. All decompressed Class A, Class B, and Class C data are archived in this database for generating historical trend retrospectives of operation reports, energy efficiency analysis reports, and fault diagnosis.

[0078] The data return flowchart of this invention is attached. Figure 1 As shown, the system architecture based on the synergy of data value extraction, transmission efficiency optimization, and reliability assurance is clearly revealed.

[0079] The core objective of this invention is to minimize total bandwidth consumption while maximizing complete data transmission. To this end, this application first divides the raw ship operation data into Class A real-time data and Class C terminal data based on data backhaul classification rules. Then, it performs dynamic network transmission according to the data transmission mechanism of Class A real-time data, effectively reducing the invalid transmission of non-critical data from the source. This effectively solves the core contradictions between massive data and limited transmission channels, real-time services and unreliable networks, and heterogeneous data and unified backhaul in the LNGC digital twin scenario.

[0080] Meanwhile, this invention employs the GZip lossless compression algorithm, significantly reducing the data load that needs to be transmitted back via satellite. Furthermore, through the reliable transmission strategy of the MQTT reliable transmission protocol and the intelligent network disconnection and resume transmission mechanism, it significantly reduces the dependence on satellite bandwidth while ensuring 100% complete data transmission. This forms a closed-loop technical system of "data acquisition - value extraction - efficient compression - reliable transmission," systematically overcoming the key technical obstacles to the efficient, complete, and reliable transmission of massive amounts of high-fidelity ship operation data under limited satellite bandwidth conditions. This lays a solid technical foundation for engineering applications in the construction of high-fidelity LNGC digital twins.

[0081] It should be understood that the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

Claims

1. A method for LNGC ship-to-shore data compression and efficient backhaul, characterized in that, Specifically, the following steps are included: S1: Collect multi-source heterogeneous raw ship operation data from the LNG ship's own IAS system and preprocess the raw data; S2: Based on the set data feedback classification rules, the preprocessed raw ship operation data is divided into Class A real-time data and Class C terminal data; S3: Determine the dynamic attributes of the real-time data of type A. If the real-time data of type A is trigger-type data and / or polling-type data that needs to be returned in real time, then compress the real-time data of type A efficiently according to the transmission mechanism and return it to the shore in a timely manner through the current network. If the real-time data of type A is other polling data, then determine whether the current communication network is a 5G network. If the current communication network is a 5G network, then compress the data efficiently and transmit it back to the shore in a timely manner through the current communication network. If the current communication network is a satellite network, then first compress the data efficiently, then downgrade the compressed data packet and mark it as a type B historical data packet and store it locally on the ship. When the communication network switches to a 5G network, then transmit this type of data back to the shore. S4: Store Class C terminal data directly on the ship's local machine, and then transmit it back to the shore in batches after the ship docks.

2. The LNGC ship-to-shore data compression and efficient backhaul method according to claim 1, characterized in that: In step S1, when preprocessing the raw ship operation data, a time series processing method based on a sliding window is first used to filter out noise and perform deduplication, and then the statistical values ​​of the data are calculated based on the set time interval.

3. The LNGC ship-to-shore data compression and efficient backhaul method according to claim 1, characterized in that: In step S2, when the preprocessed raw ship operation data is divided into Class A real-time data and Class C terminal data according to the set data feedback classification rules, Class A real-time data includes three important dynamic data: equipment status, cargo hold safety monitoring and safety alarms, and adopts a transmission trigger strategy of reporting when there is a change. Class C terminal data includes equipment operation logs, historical alarm records and conventional process parameter data.

4. The LNGC ship-to-shore data compression and efficient backhaul method according to claim 1, characterized in that: In step S3, if data transmission fails or the current communication network is interrupted during the process of transmitting Class A real-time data back to the shore, the compressed data packet is downgraded and marked as a Class B historical data packet and stored locally on the ship. After the communication network is restored, the stored Class B historical data packets are transmitted back to the shore first.

5. The LNGC ship-to-shore data compression and efficient backhaul method according to claim 1, characterized in that: In step S3, the GZip lossless compression algorithm is used to compress the data efficiently.

6. The LNGC ship-to-shore data compression and efficient backhaul method according to claim 1, characterized in that: In step S3, the QoS 1 or QoS 2 quality of service level is configured for the data packets of Class A real-time data and Class B historical data data packets through the MQTT protocol, and the compressed data packets are published to the communication network.

7. The LNGC ship-to-shore data compression and efficient backhaul method according to claim 1, characterized in that: In step S3, after the ship docks, the Class C terminal data is transmitted to the shore in batches via the port's wireless network.

8. A ship-to-shore data compression and high-efficiency backhaul system based on the method of any one of claims 1-7, characterized in that, This includes shipboard data processing systems deployed on vessels and shore-based data processing systems deployed on shore. The ship-side data processing system is used to collect multi-source heterogeneous raw ship operation data from the IAS system of LNG ships and preprocess the collected data. Then, according to the set data return classification rules, the preprocessed raw ship operation data is divided into Class A real-time data and Class C terminal data. Based on the dynamic attributes and transmission mechanism of Class A real-time data, its dynamic network transmission strategy is determined for return to the shore. Class C terminal data is stored directly on the ship and returned to the shore in batches after the ship docks. Class A data packets that fail to be transmitted in real time are downgraded and marked as Class B historical data packets and stored on the ship for retransmission. The shore-based data processing system is used to subscribe to, receive, and decompress compressed data sent by the ship-based data processing system. It pushes the decompressed data to the shore-based digital twin application platform in real time, and simultaneously stores all decompressed data in the shore-based database.

9. The ship-to-shore data compression and high-efficiency backhaul system according to claim 8, characterized in that, The shipboard data processing system includes a data acquisition module, a data processing module, a data compression module, and a data transmission and security module. The data acquisition module is used to collect multi-source heterogeneous raw ship operation data from the IAS system of LNG ships; The data processing module preprocesses the raw ship operation data. First, it categorizes the preprocessed raw data into Class A real-time data and Class C dock data according to predefined data return classification rules. Then, based on the attributes of Class A real-time data and the dynamic transmission network, it selects an appropriate transmission strategy, i.e., whether to store some data as Class B historical data on the ship. Finally, according to the dynamic network transmission strategy, it transmits Class A real-time data and Class B historical data to the data transmission and support module, while storing Class C dock data locally on the ship. In the event of data transmission failure or a detected communication network interruption, the compressed data packets that were not successfully transmitted in real-time are downgraded and marked as Class B historical data packets and stored locally on the ship. Once the communication network is restored, these data packets are prioritized for transmission back to the shore. The data compression module is used to compress the Class A real-time data that needs to be transmitted back to the shore using the GZip lossless compression algorithm based on the DEFLATE format, and then transmit the compressed package to the data transmission and protection module. The data transmission and security module is used to transmit data packets of Class A real-time data and Class B historical data back to the shore in a timely manner through the corresponding communication network based on the MQTT protocol's service classification strategy.

10. The ship-to-shore data compression and high-efficiency backhaul system according to claim 8, characterized in that, The shore-based data processing system includes a data receiving and decompression module and a data distribution and application module. The data receiving and decompression module is used to subscribe to and receive compressed data sent by the ship's data processing system, reply with confirmation messages according to the MQTT protocol specification, and decompress the successfully received data using GZip. The data distribution and application module is used to push the decompressed data to the onshore digital twin application platform in real time for 3D visualization and alarm analysis, and to synchronously store all decompressed data in the onshore database.

Images