Data compression method in ship ocean operation to-shore data transmission process

Abstract

The invention relates to the technical field of ship information, and discloses a data compression method in a ship ocean operation opposite-shore data transmission process, which comprises the following steps: step 1, collecting ship multi-source sensor data, and classifying the data into switching value data and analog quantity data through a data point location dictionary table; step 2, lossless compression is carried out on the switching value data by adopting an improved LZ4-HC algorithm, lossy compression is carried out on the analog quantity data by adopting a zstd-fast algorithm, and effective data bits are reserved according to a preset precision threshold value; and step 3, monitoring the bandwidth, delay and packet loss rate of the satellite link in real time, and dynamically adjusting the compression ratio according to the link quality. Through a mixed compression strategy, compression algorithms are automatically switched according to data types, an improved LZ4-HC high-compression-ratio mode is adopted for a control instruction, and a zstd-fast low-delay mode is adopted for sensor data, so that the compression ratio of the control instruction is increased by 25%, the delay of the sensor data is reduced by 30%, and the comprehensive compression efficiency is increased by more than 60%.

Description

Technical Field

[0001] The present invention relates to the field of ship information technology, and in particular to a method for compressing data during the transmission of data from a ship in ocean-going operation to a shore. Background Art

[0002] Because ships operating overseas need to communicate with shore using satellite networks, communication costs are high. To address this issue, data compression is often used to reduce communication costs. Currently, commonly used compression algorithms are divided into lossless and lossy. Lossless compression algorithms are based on dictionary-based string matching and replacement techniques, utilizing hash tables to accelerate the search for repeated patterns and achieving compression by storing reference locations of matching items. Lossy compression algorithms combine LZ77, Huffman coding, and high-order permutation algorithms, dynamically adjusting dictionaries and encoding strategies to balance compression ratio and speed.

[0003] However, lossless compression algorithms have a low compression ratio. In scenarios requiring a high compression ratio, the compression ratio of lossless compression algorithms is significantly lower than that of lossy compression algorithms, resulting in wasted storage space. Furthermore, for data with a large dynamic range, such as soil moisture content, fixed compression strategies cannot balance accuracy and compression ratio. The high compression level of lossy compression algorithms leads to a surge in CPU usage, impacting the multitasking capabilities of the onboard gateway. Decompression consumes a lot of energy, resulting in low compression efficiency for small data packets and unstable compression delays for short message control commands. Furthermore, their fixed compression ratio cannot balance the needs of low-bandwidth and high-latency scenarios, and they lack multi-satellite link redundancy, making it impossible to quickly switch when a single link fails, resulting in data transmission interruption. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides a data compression method during the data transmission process of a ship's ocean-going operations to the other side, which solves the problems of high transmission costs caused by low compression ratio, high latency affecting real-time performance, and data packet loss caused by network fluctuations.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for compressing data during data transmission from a ship in ocean-going operations to a shore, comprising the following steps:

[0006] Step 1: Collect multi-source sensor data from the ship and classify the data into switch data and analog data through the data point dictionary table;

[0007] Step 2: Perform lossless compression on the switch data using the improved LZ4-HC algorithm, perform lossy compression on the analog data using the zstd-fast algorithm, and retain valid data bits according to a preset accuracy threshold;

[0008] Step 3: Monitor the bandwidth, delay, and packet loss rate of the satellite link in real time, and dynamically adjust the compression ratio based on the link quality;

[0009] Step 4: Transmit compressed data through multi-path redundancy. When a single link fails, switch to the backup link and give priority to transmitting high-priority data.

[0010] Preferably, the data classification includes:

[0011] The switch data is defined as the equipment status enumeration value, including 0 / 1 type control instructions and equipment start and stop signals. The switch data includes: starboard drag head flushing gate valve (through outboard HV17) - opening, 1# mud tank flushing butterfly valve HBV02 - closing, left drag center A-frame - releasing action in progress, left 10# large mud door - opening, right mud pump clutch - low speed clutch engaging, left mud pump clutch - low speed disengaging, etc.

[0012] Analog data is defined as double-precision floating-point sensor data, including environmental parameters and equipment operating indicators. This includes: right main engine camshaft speed, right main engine intercooler inlet low-temperature water pressure, right main engine B1 cylinder liner temperature (point 1), left propulsion gearbox thrust bearing temperature, right propulsion gearbox bearing temperature (behind the output shaft), right mud pump power, right main engine charge air pressure, left allowable power, and 9# hydraulic pump pressure.

[0013] Preferably, the lossy compression includes:

[0014] Perform time domain differential compression on continuously changing GPS coordinate data, and only transmit the changes in adjacent data; the changes in adjacent data are the changes in continuous coordinates. Rather than the complete coordinates. For example:

[0015] Original coordinates: Lat = 34.5678°, Lon = 118.1234°

[0016] Change: ΔLat = 0.0001°, ΔLon = 0.0002°

[0017] The vibration monitoring data is compressed in the frequency domain, and the main frequency components are retained and high-frequency noise is filtered through FFT transformation.

[0018] Predefine the frequency range based on the signal characteristics. For example:

[0019] The reamer bearing vibration signal retains the 0-200Hz component and filters out high-frequency noise.

[0020] The underwater pump pressure signal retains the 0-50Hz component and the main fluctuation characteristics.

[0021] Specific steps of lossy compression:

[0022] 1. Convert analog data into double-precision floating-point array;

[0023] 2. Precision control: Batch modify the data in the double-precision array according to the preset threshold (such as retaining two decimal places);

[0024] 3. When the network link is high-quality, use zstd-fast mode (low latency, low compression ratio). Or, when the network link is low-quality, set the compressionLevel parameter to 15-18 to increase compression time and improve compression ratio.

[0025] Redundancy elimination:

[0026] 1. Time domain difference: only the change in continuous GPS coordinates is transmitted (such as latitude difference longitude difference Δλ).

[0027] 2. Frequency domain compression: Perform FFT transformation on vibration monitoring data (such as reamer bearing vibration waveform) to retain the main frequency component.

[0028] Preferably, the lossless compression and lossy compression are hardware accelerated by an FPGA chip, specifically including:

[0029] Using Xilinx UltraScale+ series FPGAs, the Huffman coding algorithm was converted into hardware logic using the Vivado HLS tool;

[0030] Configure the DMA controller to directly transfer compressed data to the gateway buffer to reduce CPU interrupts.

[0031] Preferably, the processing of the high priority data includes:

[0032] Safety-critical data are marked as high priority according to the data point dictionary table, including equipment failure alarms and collision warning signals; high priority data: collision warning, large mud door - blocked, main engine lubricating oil inlet temperature - high alarm, main engine turbocharger A lubricating oil pressure - low alarm, engine B column exhaust average temperature - high alarm, main engine starting air pressure - low alarm, etc.

[0033] According to the pre-established data point dictionary table, in addition to distinguishing whether the data type reported by the point is a switch quantity or an analog quantity, it will also mark whether the point data is high-priority data and the emergency sending threshold. According to the data point dictionary and the gateway reported data, a program is written through Java code to compare them. When the high-priority data is equal to the emergency sending threshold, the program skips the compression module and directly sends the high-priority data.

[0034] When the network is congested, high-priority data is transmitted directly, bypassing the compression module, and non-critical data is temporarily stored in the message queue.

[0035] Connecting to the ship's engine room dynamic environment system via the Socket protocol allows real-time link bandwidth data to be obtained. Network congestion is identified when the ratio of current bandwidth usage to total bandwidth exceeds 95%. During network congestion, other transmitted data is temporarily stored in a message queue, prioritizing high-priority data and reducing latency for critical information.

[0036] Preferably, the specific method of dynamically adjusting the compression ratio is:

[0037] When the link bandwidth is greater than 5Mbps and the packet loss rate is less than 2%, the low compression ratio mode is enabled to prioritize real-time performance. The compression ratio of the low compression ratio mode is 2:1.

[0038] When the link bandwidth is less than 1Mbps and the packet loss rate is greater than 5%, the high compression ratio mode is enabled to prioritize data integrity. The compression ratio of the low compression ratio mode is 10:1.

[0039] Preferably, the compression method further comprises:

[0040] Data framing: splitting the data stream into 512-byte key frames, assigning each frame a unique increasing sequence number;

[0041] Retransmission mechanism: When the receiving end detects discontinuity in the frame sequence, it sends a NACK request for retransmission. The sending end triggers retransmission of lost frames based on the timeout timer.

[0042] Preferably, the deployment of the FPGA chip meets the following conditions:

[0043] The interface specification supports PCIe Gen3 x8 or SATA 3.0 to communicate with the gateway;

[0044] Environmental adaptability complies with IEC 60529 IP67 protection level, operating temperature ≤ 85℃.

[0045] Preferably, the multipath redundant transmission includes:

[0046] Connect at least two satellite links simultaneously and detect link status in real time;

[0047] When the packet loss rate of the primary link exceeds the threshold or the delay is greater than 300ms, data transmission is switched to the backup link.

[0048] Preferably, the compression method further includes abnormal data processing:

[0049] Enable local caching mechanism for compressed data that fails to be transmitted, with a caching time of ≥ 24 hours;

[0050] When the link is restored, the unacknowledged key frame data in the cache is retransmitted first.

[0051] The present invention provides a method for compressing data during the transmission process of ocean-going vessel data to shore. It has the following beneficial effects:

[0052] This paper uses a hybrid compression strategy, automatically switching compression algorithms based on data type. It employs an improved LZ4-HC high-compression mode for control instructions and a zstd-fast low-latency mode for sensor data. This improves the control instruction compression ratio by 25%, reduces sensor data latency by 30%, and improves overall compression efficiency by over 60%. Furthermore, an FPGA hardware-accelerated compression module reduces end-to-end latency to <50ms.

[0053] Through dynamic compression strategies, bandwidth fluctuations are detected in real time, improving transmission efficiency by 40%-60%. Simultaneously connecting to two communication satellites, the system monitors the quality of both links in real time and dynamically allocates data streams. In the event of a single link failure, data transmission interruption time is reduced from an average of 45 seconds to less than 5 seconds, improving transmission reliability by 90%. Furthermore, safety-critical data is transmitted directly, bypassing compression to ensure zero-latency response. During network congestion, low-priority data is temporarily stored in a message queue, reducing the packet loss rate for critical data to 0.1%. Through coordinated optimization of software and hardware, the system achieves a balance between data compression efficiency, real-time transmission, and communication costs in ocean-going scenarios, while also improving system reliability and environmental adaptability, providing an efficient and low-cost communication solution for intelligent ship operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of the method of the present invention;

[0055] Figure 2 This is a flow chart of the network adaptation mechanism in the present invention. DETAILED DESCRIPTION

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0057] Please see the attached Figure 1 -Attached Figure 2 The embodiment of the present invention provides a method for compressing data during the transmission of data from a ship operating in the ocean to a shore, comprising the following steps:

[0058] Step 1:

[0059] Collect multi-source sensor data from ships and classify the data into switch data and analog data through the data point dictionary table.

[0060] Data from multiple sensors is collected via the RS485 port via the CAN bus or Modbus TCP protocol and aggregated to the shipboard gateway. A lightweight compression module is deployed in the gateway, and an FPGA is used to offload the main controller load. These sensors include: cooling drive motor oil pressure, cooler outlet oil temperature, trunnion draft, cutter depth, cutter motor current, cutter speed, mud pump speed, submersible pump motor power, mud discharge pipeline flow rate, and submersible pump power.

[0061] Deployment process:

[0062] 1. Chip selection: Choose FPGA (such as Xilinx, Zynq, UltraScale+) based on the ship operation scenario

[0063] Need to meet:

[0064] Computing capability: supports processing ≥1Gbps data stream per second;

[0065] Interface specification: Provides PCIe Gen3 x8 or SATA 3.0 interface to communicate with the gateway host controller;

[0066] Environmental adaptability: Complies with IEC 60529 IP67 protection level (anti-salt spray corrosion).

[0067] 2. Physical installation:

[0068] Adopting the design of cold plate + heat sink fins to ensure the chip operating temperature is ≤85℃;

[0069] It is integrated with the gateway motherboard through the M.2 interface or PCIe slot and supports hot-swap maintenance.

[0070] 3. Lightweight compression module deployment

[0071] The compression algorithm (Huffman coding) is integrated inside the FPGA. The Huffman coding compression algorithm is first implemented through C language programming, and the Vivado HLS tool is used to convert the C language algorithm into hardware description language (VHDL) and embed it into the FPGA.

[0072] Configuring the DMA controller for direct memory access to avoid CPU interrupts:

[0073] FPGA chips can implement compression algorithms using hardware description languages (Verilog / VHDL) and support dynamic reconfiguration. Their parallel computing capabilities significantly increase throughput, making them suitable for processing high-frequency sensor data.

[0074] A lightweight FPGA-based compression module is deployed within the shipboard gateway. The main controller (such as a CPU) sends raw data via the PCIe bus or a high-speed serial interface to the FPGA chip, where dedicated hardware executes the compression algorithm. This process directly processes data through hardware logic, eliminating the need to utilize the main controller's (CPU) computing power.

[0075] By configuring DMA transfer, a dedicated memory channel can be established to write compressed data directly into the gateway buffer, effectively reducing CPU intervention time by 80%.

[0076] Huffman coding compression algorithm

[0077] Ship equipment status logs have strong regularity and high repeatability:

[0078] Typical data: includes device start and stop records (such as "Pump_Start", "Pump_Stop"), status codes (such as "Normal_Operation, Valve_Open / Close, Warn, Error_0x12"), timestamps, etc.

[0079] Frequency distribution: Approximately 65% of device states are normal (e.g., "Normal_Operation"), 20% are periodic (e.g., "Valve_Open / Close"), and 15% are abnormal (e.g., "Error_0x12, Warn")

[0080] When processing device status logs, we first count the frequency of occurrence of each status message and construct a Huffman tree based on the frequency. We ensure that nodes with higher probabilities are closer to the root node. Then, using the Huffman tree, we convert the status information into variable-length codes, using shorter codes for more frequent characters and longer codes for less frequent characters, thus achieving data compression.

[0081] The frequency statistics table is as follows:

[0082] Status indicator Frequency Original encoding Huffman coding Normal_Operation 65％ 8bit 01 Valve_Open / Close 20％ 8bit 101 Error 10％ 8bit 1100 Warn 5％ 8bit 11100

[0083] Result:

[0084] Compression latency: After offloading from the main controller, end-to-end compression latency is reduced to <50ms (traditional solutions are >150ms)

[0085] Energy consumption optimization: The power consumption of the dedicated chip is only 20%-30% of that of the CPU solution, extending the life of ship equipment.

[0086] Power consumption comparison:

[0087] FPGA solution: 12W

[0088] CPU solution: 45W

[0089] Improved endurance: With a 500kW marine power station configuration, the operating time is expected to be extended by 2.3%-3.1%.

[0090] The gateway reports to the shipboard server through the MQTT protocol. After the shipboard server receives the data, it classifies the data into switch data and analog data according to the data type. Analog data is double-precision numerical type data, and switch data is 0 or 1 type enumeration data.

[0091] Distinguishing data types (switching data and analog data) plays an important role in data processing and analysis, which is reflected in the following aspects:

[0092] 1. Data processing methods: Different types of data require different processing methods. For example, switching data is often used to represent the on / off state of a device and may only require simple logical judgment; whereas analog data is used to represent continuously changing physical quantities and requires more complex mathematical operations and statistical analysis.

[0093] 2. Data Accuracy and Range: Digital data has only two states, 0 and 1, and has low precision. Analog data, on the other hand, can be continuous values within an arbitrary range and has higher precision. Therefore, when processing and analyzing data, it is necessary to select the appropriate precision and range based on the data type.

[0094] 3. Data application scenarios: Different types of data are suitable for different application scenarios. For example, in equipment monitoring and fault diagnosis, digital data may be used to trigger alarms or control the start and stop of equipment; while analog data may be used to analyze the operating status and performance trends of equipment.

[0095] A data point dictionary table is established in advance to define whether the data type reported by the point is a switch quantity or an analog quantity. A program is written in Java code to compare the data point dictionary and the data reported by the gateway to perform data classification.

[0096] Switching data include: starboard dredge head flushing valve (through outboard HV17) - opening, 1# mud tank flushing butterfly valve HBV02 - closing, left dredge middle A-frame - releasing action in progress, left 10# large mud door - opening, right mud pump clutch - low speed clutch engaging, left mud pump clutch - low speed disengaging, etc.

[0097] The analog data include: right main engine camshaft speed, right main engine intercooler inlet low-temperature water pressure, right main engine B1 cylinder liner temperature point 1, left propulsion gearbox thrust bearing temperature, right propulsion gearbox bearing temperature (behind the output shaft), right mud pump power, right main engine boost air pressure, left allowable power, 9# hydraulic pump pressure, etc.

[0098] Step 2:

[0099] The improved LZ4-HC algorithm is used for lossless compression of switch data, and the zstd-fast algorithm is used for lossy compression of analog data, and the valid data bits are retained according to the preset accuracy threshold.

[0100] Among them, lossless compression: write a Java program and use the LZ4-HC (high compression ratio mode) algorithm to perform lossless compression on the switch quantity to ensure zero-error transmission of data.

[0101] Specific steps for lossless compression:

[0102] 1. Convert the switch data into a binary array. Example: byte[]input = {0x01, 0x00, 0x01, 0x01, 0x00};

[0103] 2. When the network link is low-quality, use LZ4-HC mode (high compression ratio), sacrificing some compression speed in exchange for a higher compression rate. When the network link is high-quality and bandwidth transmission costs are not a concern, use LZ4-fast mode, sacrificing some compression ratio in exchange for faster compression speed.

[0104] Key operations:

[0105] Initialize the compressor highCompressor to a high compression ratio and fastCompressor to a low compression ratio:

[0106] LZ4Factory.highCompressor() / LZ4Factory.fastCompressor()

[0107] Calculate the maximum compressed length: compressor.maxCompressedLength(input.length)

[0108] Perform compression: compressor.compress(input, 0, input.length, compressed, 0, maxCompressedLength)

[0109] Output format: The compressed data is a binary stream, which can be directly stored as a file or transmitted over the network.

[0110] Lossy compression: Write a Java program that uses the zstd-fast (low-latency mode) algorithm to perform lossy compression on sensor data (such as soil moisture content, vibration monitoring data, pipeline pressure, pipeline flow rate concentration, etc.), retaining the main features of the data based on the preset accuracy threshold.

[0111] Specific steps of lossy compression:

[0112] 1. Convert the analog data into a double-precision floating-point array, for example: double[]sensorData={12.345,23.456,34.567,45.678};

[0113] 2. Precision control: Batch modify the data in the double-precision array according to the preset threshold (such as retaining two decimal places). The modified result is as follows: double[]sensorData={12.34,23.45,34.56,45.67};

[0114] 3. When the network link is high-quality, use zstd-fast mode (low latency, low compression ratio). Or, when the network link is low-quality, set the compressionLevel parameter to 15-18 to increase compression time and improve compression ratio.

[0115] Key operations:

[0116] Initialize the compressor: Zstd.compress(data,compressionLevel)

[0117] Set compression level: 1-22

[0118] Output format: The compressed data is a binary stream and can be directly stored or transmitted.

[0119] Redundancy elimination:

[0120] 1. Time domain difference: only the change in continuous GPS coordinates is transmitted (such as latitude difference longitude difference Δλ).

[0121] 2. Frequency domain compression: Perform FFT transformation on vibration monitoring data (such as reamer bearing vibration waveform) to retain the main frequency component.

[0122] Specific steps for redundancy elimination:

[0123] 1. Time domain difference (for continuously changing data)

[0124] GPS coordinate compression: only transmit the difference between consecutive coordinates Not the full coordinates.

[0125] For example:

[0126] Original coordinates: Lat = 34.5678°, Lon = 118.1234°

[0127] Change: ΔLat = 0.0001°, ΔLon = 0.0002°

[0128] 2. Frequency domain compression (for vibration / waveform data)

[0129] FFT transformation and main frequency retention: Perform FFT transformation on the vibration waveform, retain the main frequency component (such as 0-100Hz), and filter high-frequency noise.

[0130] Predefine the frequency range based on the signal characteristics. For example:

[0131] The reamer bearing vibration signal retains the 0-200Hz component and filters out high-frequency noise.

[0132] The underwater pump pressure signal retains the 0-50Hz component and the main fluctuation characteristics.

[0133] Step 3:

[0134] Monitor the bandwidth, delay, and packet loss rate of satellite links in real time, and dynamically adjust the compression ratio based on link quality.

[0135] Among them, a network adaptive mechanism is used to dynamically adjust the compression ratio. That is, a Java program is written to obtain bandwidth perception data by connecting to the dynamic environment system of the computer room, monitor the link quality (network bandwidth, delay and packet loss rate) in real time, and dynamically adjust the compression ratio.

[0136] High bandwidth mode (>5Mbps): Enables low compression ratio (2:1) to ensure real-time performance.

[0137] Low bandwidth mode (<1Mbps): Enables a high compression ratio (10:1) to prioritize data integrity.

[0138] Technical advantages: Compared with fixed compression ratio solutions, it can improve transmission efficiency by 40%-60%.

[0139] Priority Queue:

[0140] Classification marking: Mark safety-related data (such as collision warning) as high priority and transmit it directly without compression.

[0141] High-priority data: collision warning, large mud door - blocked, main engine lubricating oil inlet temperature - high alarm, main engine turbocharger A lubricating oil pressure - low alarm, engine B row exhaust average temperature - high alarm, main engine starting air pressure - low alarm, etc.

[0142] According to the pre-established data point dictionary table, in addition to distinguishing whether the data type reported by the point is a switch quantity or an analog quantity, it will also mark whether the point data is high-priority data and the emergency sending threshold. According to the data point dictionary and the gateway reported data, a program is written through Java code to compare them. When the high-priority data is equal to the emergency sending threshold, the program skips the compression module and directly sends the high-priority data.

[0143] Resource allocation: When the network is congested, other transmission data will be temporarily stored in the message queue, and high-priority data will be transmitted first to reduce the delay of critical information.

[0144] Connecting to the ship's engine room dynamic environment system via the Socket protocol allows real-time link bandwidth data to be obtained. Network congestion is identified when the ratio of current bandwidth usage to total bandwidth exceeds 95%. During network congestion, other transmitted data is temporarily stored in a message queue, prioritizing high-priority data and reducing latency for critical information.

[0145] Multi-path redundant transmission:

[0146] Dual-link backup: Connect to two communication satellites simultaneously, monitor the quality of dual links in real time, and dynamically allocate data streams.

[0147] Improved reliability: When a single link fails, data transmission interruption time is shortened from an average of 45 seconds to less than 5 seconds.

[0148] The detailed steps of bandwidth sensing data are as follows:

[0149] Write a Java program to use the socket protocol to connect to the dynamic environment system in the ship's engine room (the engine room is used to monitor the performance, bandwidth, network speed, disk space, and information of routers, switches and other ports of each server in real time, which belongs to the basic environment) to obtain real-time data such as ship network bandwidth, latency and packet loss rate.

[0150] The link quality classification standards are as follows:

[0151]

[0152] When the Java-based socket client program detects changes in link quality within the computer room's dynamic environmental system, it writes the current link quality to a public cache server. The compression processing module reads the link status from memory in real time and, based on this status, determines whether to use high compression ratio mode or high real-time mode.

[0153] Step 4:

[0154] Compressed data is transmitted through multi-path redundancy. When a single link fails, it switches to the backup link and gives priority to transmitting high-priority data.

[0155] The compressed data is sent via a satellite link, and the shore end uses a corresponding algorithm to reversely decompress it.

[0156]

[0157] In ocean voyage scenarios, data compression efficiency is increased by more than 60% and transmission costs are reduced by 45%.

[0158] Exception handling: If data loss is detected, the key frame can be automatically requested to be retransmitted.

[0159] Data framing and sequence number marking

[0160] Operation: In the compression processing module, the continuous data stream to be transmitted is divided into key frames of fixed length (such as 512 bytes per frame), and a unique increasing sequence number (such as a 32-bit unsigned integer) is assigned to each frame.

[0161] Purpose: To ensure that the receiving end can detect the continuity between frames through frame boundary division and sequence number identification.

[0162] Sender status tracking

[0163] operate:

[0164] The sender creates a data table to record the sequence numbers of frames that have been sent but not confirmed.

[0165] When each frame is sent, a timeout timer is started by the Java program (the timeout threshold is dynamically adjusted according to the current link quality, such as 300ms for high-quality links and 1500ms for low-quality links).

[0166] Purpose: To monitor the transmission status of frames and provide a basis for timeout retransmission.

[0167] Receiver confirmation mechanism

[0168] operate:

[0169] The shore-side receiver adopts a cumulative confirmation strategy. After receiving N consecutive frames, it sends an ACK packet containing the sequence number of the latest correctly received frame.

[0170] If it is detected that the frame sequence number is discontinuous (for example, sequence number 7 is received directly after 5), a NACK packet is immediately sent to request retransmission of the missing frame.

[0171] Purpose: To explicitly notify the sender of data loss through the ACK / NACK feedback mechanism.

[0172] Timeout retransmission trigger

[0173] operate:

[0174] After the sender's timeout timer expires, if no ACK for the corresponding frame is received, the frame is considered lost.

[0175] Retransmission strategy: Only retransmit lost frames (rather than all unconfirmed frames) to reduce redundant transmission.

[0176] Purpose: To compensate for data loss caused by high packet loss rate in low-quality links (such as heavy rain).

[0177] In summary, this embodiment:

[0178] A hybrid compression strategy is adopted, combining lossless / lossy algorithms to adapt to the multi-type data characteristics of ships' ocean-going operations.

[0179] Dynamic network adaptation adjusts network parameters in real time through bandwidth perception to solve the problem of ocean network fluctuations.

[0180] Differential compression optimization improves the compression rate by 40%-60% for continuously changing data such as sensors and GPS.

[0181] Compared with existing solutions:

[0182]

[0183] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for compressing data during the transmission of data from a ship to shore during ocean-going operations, characterized in that: The following steps are involved: Step 1: Collect multi-source sensor data from the ship and classify the data into switch data and analog data through the data point dictionary table; Step 2: Perform lossless compression on the switch data using the improved LZ4-HC algorithm, perform lossy compression on the analog data using the zstd-fast algorithm, and retain valid data bits according to a preset accuracy threshold; Step 3: Monitor the bandwidth, delay, and packet loss rate of the satellite link in real time, and dynamically adjust the compression ratio based on the link quality; Step 4: Transmit compressed data through multi-path redundancy. When a single link fails, switch to the backup link and give priority to transmitting high-priority data.

2. The method for compressing data during the transmission of data from a ship operating in the ocean to the shore according to claim 1, characterized in that: The data categories include: Define switch data as device status enumeration values, including 0 / 1 type control instructions and device start / stop signals; Analog data is defined as double-precision floating-point sensor data, including environmental parameters and equipment operation indicators.

3. The method for compressing data during the transmission of data from a ship operating in the ocean to the shore according to claim 1, characterized in that: The lossy compression includes: Perform time domain differential compression on continuously changing GPS coordinate data, and only transmit the changes in adjacent data; The vibration monitoring data is compressed in the frequency domain, and the main frequency components are retained and high-frequency noise is filtered through FFT transformation.

4. The method for compressing data during the transmission of data from a ship operating in the ocean to the shore according to claim 1, characterized in that: The lossless compression and lossy compression are hardware accelerated by FPGA chips, specifically including: Using Xilinx UltraScale+ series FPGAs, the Huffman coding algorithm was converted into hardware logic using the Vivado HLS tool; Configure the DMA controller to directly transfer compressed data to the gateway buffer to reduce CPU interrupts.

5. The method for compressing data during the transmission of data from a ship operating in the ocean to the shore according to claim 1, characterized in that: The processing of the high priority data includes: Mark safety-critical data as high priority based on the data point dictionary table, including equipment failure alarms and collision warning signals; When the network is congested, high-priority data is transmitted directly, bypassing the compression module, and non-critical data is temporarily stored in the message queue.

6. The method for compressing data during the transmission of data from a ship operating in the ocean to a shore according to claim 1, characterized in that: The specific method of dynamically adjusting the compression ratio is: When the link bandwidth is greater than 5Mbps and the packet loss rate is less than 2%, the low compression ratio mode is enabled to prioritize real-time performance. The compression ratio of the low compression ratio mode is 2:

1. When the link bandwidth is less than 1Mbps and the packet loss rate is greater than 5%, the high compression ratio mode is enabled to prioritize data integrity. The compression ratio of the low compression ratio mode is 10:

1.

7. The method for compressing data during the transmission of data from a ship operating in the ocean to a shore according to claim 1, characterized in that: The compression method further comprises: Data framing: splitting the data stream into 512-byte key frames, assigning each frame a unique increasing sequence number; Retransmission mechanism: When the receiving end detects discontinuity in the frame sequence, it sends a NACK request for retransmission. The sending end triggers retransmission of lost frames based on the timeout timer.

8. The method for compressing data during the transmission of data from a ship operating in the ocean to a shore according to claim 4, characterized in that: The deployment of the FPGA chip meets the following conditions: The interface specification supports PCIe Gen3 x8 or SATA 3.0 to communicate with the gateway; Environmental adaptability complies with IEC 60529 IP67 protection level, and the operating temperature is ≤85℃.

9. The method for compressing data during the transmission of data from a ship operating in the ocean to a shore according to claim 1, characterized in that: The multi-path redundant transmission includes: Connect at least two satellite links simultaneously and detect link status in real time; When the packet loss rate of the primary link exceeds the threshold or the delay is greater than 300ms, data transmission is switched to the backup link.

10. The method for compressing data during the transmission of data from a ship operating in the ocean to a shore according to claim 1, characterized in that: The compression method further includes abnormal data processing: Enable local caching mechanism for compressed data that fails to be transmitted, with a caching time of ≥ 24 hours; When the link is restored, the unacknowledged key frame data in the cache is retransmitted first.

Images