Edge lightweight data transmission method and system for observation monitoring terminal in polar environment

Abstract

This invention relates to a lightweight edge data transmission method and system for observation and monitoring terminals in polar environments. The method includes: at an edge node in a polar environment, identifying the device type and communication protocol of the observation and monitoring terminal, and calling the corresponding protocol conversion rules to obtain the converted raw data; acquiring the on / off status and link quality parameters of the current communication link of the edge node to obtain a comprehensive link quality score; compressing the raw data in real time to generate a lightweight data packet; comparing the comprehensive link quality score with a preset score threshold; if it is greater than or equal to the preset score threshold, the edge node transmits the lightweight data packet back to a remote data center in real time; otherwise, it stores it locally and transmits it after the current communication link is restored. Compared with the prior art, this invention can automatically switch transmission and storage modes according to link quality, improving the transmission reliability and system deployment efficiency of polar observation data.

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, and in particular to a lightweight edge data transmission method and system for monitoring terminals in polar environments. Background Technology

[0002] As polar scientific research and operational activities continue to deepen, the need for real-time acquisition and efficient management of high-quality data is becoming increasingly urgent.

[0003] However, the extreme polar environment presents severe challenges to data transmission. First, polar communication links are characterized by high latency, high packet loss, extremely limited bandwidth, and instability. Existing satellite communications, such as Iridium and BeiDou short message services, typically have bandwidths of only a few kbps to tens of kbps, and are frequently interrupted due to polar ionospheric disturbances and severe weather. Traditional data transmission methods mostly assume continuous network availability or recoverability after interruptions, making them ill-suited to the complex scenarios of prolonged network outages and intermittent weak connections in the polar environment. This leads to the easy loss of high-value observational data during transmission. Current technologies often assess link quality based on a single parameter or simple threshold, failing to consider the instantaneous and drastic fluctuations in polar links, leading to misjudgments. Furthermore, there is a lack of a mechanism to simultaneously apply link quality assessment results to compression ratio adjustments and transmission / storage decisions.

[0004] Secondly, there is a wide variety of polar observation and monitoring terminals, including weather stations, ice-based buoys, unmanned ice stations, AIS receivers, seismographs, etc., with different manufacturers and models using different communication protocols. In the absence of a unified access framework, edge nodes need to spend several hours or even longer manually adapting protocols, which severely restricts the rapid deployment capability of the observation system.

[0005] Furthermore, existing edge computing solutions have significant shortcomings in polar applications. On the one hand, general data compression algorithms have high computational complexity, making them difficult to run in real time on edge nodes with limited power consumption and computing power. On the other hand, traditional storage strategies either transmit all data back in real time, but the success rate is extremely low under weak links, or store all data locally, which can easily lead to storage overflow and data not being used in a timely manner. Moreover, when local storage space is insufficient and old data needs to be deleted, existing solutions only perform the deletion operation without notifying the remote data center, leaving the remote end completely unaware of the specific scope of data loss and unable to perform subsequent remediation or data repair. During the resume transmission process after the link is restored, existing breakpoint resume transmission mechanisms usually need to retransmit the entire file or retransmit all subsequent data from the breakpoint, which can easily cause a large number of duplicate transmissions, wasting the extremely narrow satellite bandwidth and lacking an intelligent decision-making mechanism to dynamically adjust the transmission and storage modes according to the link status.

[0006] In summary, data transmission to observation and monitoring terminals in polar environments with weak data links suffers from technical bottlenecks such as unstable transmission, slow adaptation, and data loss due to network outages. In particular, there is a lack of edge lightweight processing methods capable of accurately assessing link quality and adjusting compression ratios and transmission strategies accordingly. Therefore, there is an urgent need for an edge lightweight processing method that can automatically switch transmission and storage modes based on link quality to improve the reliability of polar observation data transmission and the efficiency of system deployment. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a lightweight edge data transmission method and system for monitoring terminals in polar environments that can automatically switch transmission and storage modes according to link quality.

[0008] The objective of this invention can be achieved through the following technical solutions: A lightweight edge data transmission method for monitoring terminals in polar environments includes the following steps: At edge nodes in polar environments, the device type and communication protocol of the monitoring terminals connected to the polar edge nodes are identified by a pre-set multi-source heterogeneous device protocol feature library and adaptive matching algorithm, so as to call the corresponding protocol conversion rules and obtain the converted raw data. The system acquires the connectivity status of the current communication link of the edge node. If the link remains active, it collects the link quality parameters of the current communication link and performs parameter filtering and normalization using a fixed or variable duration sliding time window. Then, it performs weighted fusion to obtain a comprehensive link quality score. The weighting coefficients for the weighted fusion are dynamically distributed by the remote data center or adaptively adjusted by the edge node based on historical transmission success rates. The link quality parameters include at least one of the following: signal-to-noise ratio, received signal strength indication, packet loss rate, round-trip time, and available bandwidth. The raw data is compressed in real time to generate a lightweight data packet; The link quality comprehensive score is compared with a preset score threshold. If it is greater than or equal to the preset score threshold, the edge node will transmit the lightweight data packet back to the remote data center in real time; otherwise, it will be stored locally and the unacknowledged lightweight data packet will be transmitted to the remote data center after the current communication link is restored.

[0009] Furthermore, the method also includes: The priority of the lightweight data packet is determined based on the device type of the monitoring terminal, the data sampling frequency, the urgency of the event, or the user's preset tags. When the overall link quality score is lower than a preset score threshold but higher than a preset second threshold, it is determined whether the priority of the lightweight data packet is the highest priority. The highest priority corresponds to events such as ice shock events, extreme weather events, navigation collision warning signals, and equipment failure alarms. If so, the edge node adopts a reduced bit rate, increases forward error correction coding, or adopts a modulation method with preset robustness enhancement for online transmission. Otherwise, it performs local storage. During the process of transmitting lightweight data packets to the remote data center, the edge node transmits higher-priority lightweight data packets first, followed by lower-priority lightweight data packets; for lightweight data packets of the same priority, they are sent according to the order in which the data-generating events occur.

[0010] Furthermore, the method also includes: When multiple edge nodes cover the same observation area, each edge node exchanges its own link quality comprehensive score and remaining storage capacity information through short-range wireless communication, and selects the node with the highest link quality comprehensive score and remaining storage capacity greater than the preset capacity threshold as the proxy node. The proxy node collects the indexes of untransmitted lightweight data packets stored locally by other edge nodes within the observation area, merges and deduplicates them, and then sends a unified retransmission request to the remote data center. The merging and deduplication process includes: if the difference in timestamps of data packets uploaded by two edge nodes is less than a preset time window and the numerical deviation of the same physical quantity corresponding to the same monitoring terminal is less than a preset percentage, then the data is determined to be duplicated and only one copy is retained. If a proxy node fails during transmission, the remaining edge nodes within the observation area will re-elect a new proxy node through an election mechanism and resume transmission from the point of interruption based on the records already received by the remote data center.

[0011] Furthermore, the protocol conversion rules include a field mapping table, unit conversion coefficients, and verification algorithms, which are used to uniformly convert heterogeneous data frames output by monitoring terminals from different manufacturers or types into the internal standard format of the edge nodes. The field mapping table is used to record the correspondence between source protocol fields and standard format fields; The unit conversion factor is used to uniformly convert physical quantities with different dimensions; The verification algorithm includes cyclic redundancy check.

[0012] Furthermore, the real-time compression process of the raw data includes: The numerical observation data in the original data are compressed using a lossless compression method based on differential pulse code modulation or run-length encoding. A lossy compression method based on discrete cosine transform is used to compress the image and acoustic observation data in the original data; The dynamic adjustment process of the compression ratio in the real-time compression process includes: finely adjusting the pre-set base compression ratio based on the comprehensive link quality score, the priority index of the original data, and the current remaining storage capacity of the edge nodes to obtain the dynamically adjusted compression ratio, and applying boundary constraints. The dynamic adjustment calculation expression for the compression ratio is as follows: Rc =R base ·f(LQS)·g(P)·h(S rem ) g(P) = 1 + λ·(1-P) h(S rem )=1+γ·(1-S rem ) In the formula, Rc R is the dynamically adjusted compression ratio. base The base compression ratio is f(LQS), where f(LQS) is the link quality correction factor. LQS For the comprehensive link quality score, α and β are preset parameters, g(P) is the priority correction factor, λ is the priority scaling parameter, P is the numerical result of the priority, and h(S) is the priority value. rem S is the storage urgency correction factor. rem This represents the current remaining storage capacity rate; The calculation expression for the boundary constraints is: In the formula, Rmax This represents the maximum compression ratio. Rmin This represents the minimum compression ratio.

[0013] Furthermore, the method also includes a dynamic adjustment process for the scoring threshold, which is as follows: The remote data center dynamically issues scoring thresholds based on the historical link quality comprehensive score of the current polar environment; Alternatively, the scoring threshold can be gradually lowered after a preset number of consecutive successful transmissions at the edge node, and gradually raised after a preset number of consecutive failed transmissions at the edge node.

[0014] Furthermore, the method stores lightweight data packets locally in the non-volatile storage of the edge node for local storage; if the storage space of the edge node is insufficient, the lightweight data packet with the lowest priority and the earliest generation time is deleted first, and the remote data center is notified through the out-of-band channel. The out-of-band channel is a short message service, a satellite beacon signal, or a low-frequency auxiliary communication channel; The notification to the remote data center via out-of-band channel includes the timestamp range of the deleted lightweight data packets, the packet sequence number range, and the identification of the monitoring terminal to which the data belongs.

[0015] Furthermore, the process of transmitting lightweight data packets that were not successfully acknowledged to the remote data center includes: The edge node sends a list of data packet indexes or timestamp ranges stored during the network outage to the remote data center, receives missing data packet identifiers from the remote data center, and reads the corresponding data packets from local storage based on the missing data packet identifiers for transmission. After the current communication link is restored, the method counts the duration of stable link quality. After waiting for the stable link quality to reach a preset stable duration threshold, it then transmits the lightweight data packets that were not successfully acknowledged to the remote data center. If a single lightweight data packet fails to be retransmitted multiple times, it is marked as a transmission anomaly and logged.

[0016] Furthermore, the method also includes security protection steps: The system authenticates the access monitoring terminals to reject unauthorized access; before storing and transmitting lightweight data packets, it encrypts them using a symmetric encryption algorithm and attaches a hash message authentication code; it uses an asymmetric algorithm to sign and verify the received remote control commands, executes the commands after successful verification and encrypts and sends the execution result back to the command sender, and records an exception log and triggers an alarm if the verification fails. The encryption key of the symmetric encryption algorithm is updated periodically, either through out-of-band channel distribution or through dynamic generation based on time synchronization.

[0017] The present invention also provides a lightweight edge data transmission system for monitoring terminals in polar environments, including a memory and a processor. The memory stores a computer program, and the processor calls the computer program to execute the steps of the method described above.

[0018] Compared with the prior art, the present invention has the following advantages: (1) Significantly improves data transmission reliability in polar weak link environments: This invention obtains a comprehensive link quality score through sliding time window filtering and multi-parameter weighted fusion, and simultaneously decides the data compression ratio and transmission / storage mode based on the same score. When the score is below the threshold, it automatically switches to local persistent storage. After the link is restored, only data packets that have not yet been acknowledged by the remote end are transmitted, avoiding invalid retransmissions. Unlike conventional single threshold switching, the comprehensive scoring mechanism of this invention effectively suppresses misjudgments caused by instantaneous fluctuations in polar links, ensuring zero data loss and business continuity in scenarios of long-term network outages and intermittent weak connections.

[0019] This comprehensive scoring system simultaneously controls compression ratio adjustment and transmission / storage mode switching, achieving coordinated optimization of these two key aspects, rather than simply switching independent thresholds. A precise resume mechanism for missing data packets is also included after link recovery, forming a complete closed loop.

[0020] (2) Achieving efficient management of storage resources and traceability of data loss: When local storage space is insufficient, this invention actively notifies the remote data center through an out-of-band channel independent of the main satellite link before deleting existing data packets, informing it of the timestamp range, packet sequence number, and terminal identifier of the deleted data packets. Conventional storage management only performs deletion operations without notifying the remote end, resulting in data gaps that cannot be traced; this invention enables the remote end to know the details of data loss, providing a reliable basis for subsequent manual data collection or data repair, and overcoming the technical bias of avoiding additional transmission under extremely low bandwidth.

[0021] (3) Significantly shortens the protocol adaptation time for heterogeneous devices and enhances the system's rapid deployment capability: This invention pre-installs a multi-source heterogeneous device protocol feature library and adaptive matching algorithm at polar edge nodes, automatically identifies terminal types and calls conversion rules such as field mapping, unit conversion, and verification, unifying heterogeneous data frames into an internal standard format. Protocol adaptation time is reduced from several hours to minutes, significantly improving the rapid deployment and multi-device collaboration capabilities of the polar observation system in unmanned environments.

[0022] (4) Dynamic adaptive compression to reduce bandwidth usage and storage overhead: This invention automatically selects lossless or lossy compression methods based on data type and dynamically adjusts the compression ratio according to the numerical range of the comprehensive link quality score. When the score is below the threshold, the system not only increases the compression ratio but also automatically switches to local storage. After the score recovers, the compression ratio is reduced and real-time transmission is restored. This score-driven integrated linkage surpasses conventional adaptive compression and produces a synergistic technical effect.

[0023] (5) A refined mechanism for resuming transmission after network outages, avoiding bandwidth waste: After the link is restored, the edge node sends a list of data packet indexes from the outage period to the data center, receives feedback on missing packet identifiers, and only transmits data packets that have not yet been acknowledged, prioritizing the transmission of high-priority event data (ice tremors, extreme weather, collision warnings, etc.). Compared with traditional breakpoint resumption, this invention achieves retransmission of missing data packets with precision down to the individual data packet level, avoiding bandwidth waste caused by repeated transmissions, and is especially suitable for the extremely narrow bandwidth environment of polar satellite links.

[0024] (6) Enhance system security and multi-node collaboration capabilities: Edge nodes verify the digital certificates or pre-shared keys of terminals to reject unauthorized access; asymmetric signature verification is used for remote commands to prevent tampering; symmetric encryption and message authentication codes are used for data packets to ensure confidentiality and integrity throughout the process. When multiple edge nodes cover the same area, proxy nodes are automatically elected by scoring the short-range wireless communication exchange link and storage capacity, and then merged and deduplicated before unified transmission; when a proxy node fails, it is restored through an election mechanism, which significantly improves the robustness and networking capability of the polar uninhabited area observation system. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating a lightweight edge data transmission method for observation and monitoring terminals in a polar environment, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a protocol adaptive adaptation and lightweight coding sub-process provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a link status monitoring and transmission / storage mode switching decision-making process provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0027] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0028] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0029] Example 1 like Figure 1 As shown, this embodiment provides a lightweight edge data transmission method for monitoring terminals in polar environments, including the following steps: S1: At the edge node in the polar environment, the device type and communication protocol of the monitoring terminal connected to the polar edge node are identified by a pre-set multi-source heterogeneous device protocol feature library and adaptive matching algorithm, so as to call the corresponding protocol conversion rules and obtain the converted raw data. S2: Obtain the on / off status of the current communication link of the edge node. If it remains on, collect the link quality parameters of the current communication link, and perform parameter filtering and normalization using a fixed or variable duration sliding time window. Then, perform weighted fusion to obtain a comprehensive link quality score. The weight coefficients of the weighted fusion are dynamically issued by the remote data center or adaptively adjusted by the edge node based on the historical transmission success rate. The link quality parameters include at least one of the following: signal-to-noise ratio, received signal strength indication, packet loss rate, round-trip time, and available bandwidth. S3: Uses a low-complexity coding algorithm to compress the original data in real time to generate lightweight data packets; S4: Compare the overall link quality score with the preset score threshold. If it is greater than or equal to the preset score threshold, the edge node will send the lightweight data packet back to the remote data center in real time; otherwise, it will store it locally and send the lightweight data packet that was not successfully acknowledged to the remote data center after the current communication link is restored.

[0030] Optionally, when local storage space is insufficient, the remote data center is notified via out-of-band channel before deleting existing data packets.

[0031] As a preferred technical solution of the present invention, such as Figure 2 As shown, for the acquired data stream, the system first identifies and matches the protocol to determine if the match is successful. If the match fails, the data is discarded and logged. If the match is successful, the protocol conversion rules are loaded, and subsequent processing steps are executed. The protocol conversion rules include a field mapping table, unit conversion factors, and a verification algorithm, which are used to uniformly convert heterogeneous data frames output by monitoring terminals from different manufacturers or types into the internal standard format of the edge node. The field mapping table records the correspondence between the source protocol fields and the standard format fields, the unit conversion factors uniformly convert physical quantities of different dimensions, and the verification algorithm includes cyclic redundancy check.

[0032] The adaptive matching algorithm is used to adaptively obtain the matching protocol name from the protocol feature library of multi-source heterogeneous devices based on the type of the monitoring terminal and its communication protocol through fuzzy matching.

[0033] Specifically, the low-complexity coding algorithm also includes lossless compression methods based on differential pulse code modulation or run-length encoding, as well as lossy compression methods based on discrete cosine transform. The edge nodes automatically select the compression method according to the data type: lossless compression is used for numerical observation data, and lossy compression is used for image or acoustic data. The compression ratio is dynamically adjusted according to the numerical range of the link quality comprehensive score. The lower the score, the higher the compression ratio.

[0034] Specifically, the real-time compression process of raw data includes: The lossless compression method based on differential pulse code modulation or run-length encoding is used to compress the numerical observation data in the original data. A lossy compression method based on discrete cosine transform is used to compress the image and acoustic observation data in the original data. The dynamic adjustment process of the compression ratio in real-time compression includes: finely adjusting the pre-set base compression ratio based on the comprehensive link quality score, the priority index of the original data, and the current remaining storage capacity of the edge nodes to obtain the dynamically adjusted compression ratio, and applying boundary constraints; The expression for dynamically adjusting the compression ratio is: Rc =R base ·f(LQS)·g(P)·h(S rem ) g(P) = 1 + λ·(1-P) h(S rem )=1+γ·(1-S rem ) In the formula, Rc R is the dynamically adjusted compression ratio. base The base compression ratio is f(LQS), where f(LQS) is the link quality correction factor. LQS For the comprehensive link quality score, α and β are preset parameters, g(P) is the priority correction factor, λ is the priority scaling parameter, P is the numerical result of the priority, and h(S) is the priority value. rem S is the storage urgency correction factor. rem This represents the current remaining storage capacity rate; The expression for calculating boundary constraints is: In the formula, Rmax This represents the maximum compression ratio. Rmin This represents the minimum compression ratio.

[0035] Specifically, the link quality parameters include at least one of the following: signal-to-noise ratio, received signal strength indication, packet loss rate, round-trip time delay, and available bandwidth; the length of the sliding time window is a configurable fixed duration or a variable duration, and the weighting coefficients of the weighted fusion are dynamically issued by the remote data center or adaptively adjusted by the edge node based on the historical transmission success rate.

[0036] Specifically, the preset scoring threshold is dynamically issued by the remote data center based on historical statistical data of the current polar communication environment, or adaptively adjusted by the edge node based on the statistical distribution characteristics of the comprehensive link quality score over a period of time. When the edge node successfully transmits multiple times in a row, the threshold is gradually lowered; when the edge node fails to transmit multiple times in a row, the threshold is gradually raised.

[0037] Specifically, the out-of-band channel is a short message service, a satellite beacon signal, or a low-frequency auxiliary communication channel; the notification content includes at least the timestamp range, the packet sequence number range, and the identifier of the monitoring terminal to which the data belongs.

[0038] Specifically, transmitting only lightweight data packets that the remote data center has not yet successfully acknowledged as received includes: the edge node sending a list of data packet indexes or timestamp ranges stored during the network outage to the remote data center, receiving the missing data packet identifiers from the remote data center, and reading the corresponding data packets from local non-volatile storage based on the missing data packet identifiers for transmission; if the remote data center has already obtained some data packets through other channels, the edge node skips the transmission of these data packets.

[0039] Specifically, it also includes a priority strategy: edge nodes maintain a priority queue, and after the link is restored, all data packets in the high-priority queue are sent first, followed by data packets in the low-priority queue; within the same priority, data packets are sent in the order of their generation time; priority is determined based on the monitoring terminal type, data sampling frequency, event urgency, or user-preset tags.

[0040] Specifically, when the overall link quality score is lower than the preset score threshold but higher than the preset second threshold, the edge node further determines whether the currently collected data belongs to a preset high-priority event type. High-priority event types include ice shock events, extreme weather events, navigation collision warning signals, or equipment failure alarms. If so, the edge node attempts to transmit online by reducing the bit rate, increasing forward error correction coding, or using a modulation method with preset robustness enhancement. If not, or if the overall link quality score is lower than the preset second threshold, local persistent storage is performed.

[0041] Specifically, before lightweight data packets are written to local non-volatile storage or transmitted back to the remote data center, the edge node uses a symmetric encryption algorithm to encrypt the data packets and attaches a hash-based message authentication code. The encryption key is pre-negotiated and periodically updated by the edge node and the remote data center. The update methods include out-of-band key distribution via satellite link or dynamic key generation algorithm based on time synchronization.

[0042] Specifically, after identifying the type of monitoring terminal, the edge node further verifies the terminal's digital certificate or pre-shared key, and refuses access to unauthenticated terminals. For remote control commands, the edge node uses an asymmetric signature algorithm to verify the command. The command can only be executed after the signature verification is successful, and the execution result is encrypted and sent back to the command sender. When the signature verification fails, the edge node records the abnormal access log and triggers a local alarm.

[0043] Specifically, it also includes: when multiple edge nodes cover the same observation area, each edge node exchanges its own link quality comprehensive score and remaining storage capacity information through short-range wireless communication. The node with the highest link quality comprehensive score and remaining storage capacity greater than a preset score threshold is designated as the proxy node. The proxy node collects the out-of-network cached data indexes of other nodes, merges and deduplicates them, and then uniformly initiates a resume transmission request to the remote data center. Merging and deduplication includes: if the difference in timestamps of data packets uploaded by two nodes is less than a preset window and the numerical deviation of the corresponding monitoring terminal and the same physical quantity is less than a preset percentage, it is determined to be duplicate data and only one copy is retained. If the proxy node fails during transmission, the remaining nodes re-designate a new proxy node through an election mechanism and resume transmission from the point of interruption based on the records already received by the remote data center.

[0044] The following is a specific implementation example: This example uses an unmanned ice station deployed in Prydz Bay during an Antarctic scientific expedition. This ice station is equipped with various monitoring terminals, including: Automatic weather station, outputting data such as temperature, air pressure, and humidity; Ice-based buoys output sea ice temperature profiles and GPS location; Ice tremor monitor outputs acceleration waveform; And an AIS receiver, which outputs ship movement messages.

[0045] The edge node of the ice station is a low-power embedded computer that communicates with the data center via an Iridium 9523 module. The uplink bandwidth is approximately 2.4 kbps, and the link is frequently interrupted due to ionospheric disturbances in the polar regions. It should be noted that this embodiment uses the Iridium system as an example, but the invention is not limited to this; other low-bandwidth satellite communication systems are also applicable, such as BeiDou short message service and Tiantong satellite communication.

[0046] (a) Device access and protocol adaptation After the edge node of the ice station is powered on, it runs a pre-built protocol feature library for multi-source heterogeneous devices. This feature library stores the protocol features of more than 20 common polar observation devices in JSON format, including: protocol name, start byte flag, field separator, verification algorithm type, data field definition, etc. The data field definition includes physical quantity name, data type, unit, and scaling factor.

[0047] When an automatic weather station connects to the edge node via an RS485 interface, the edge node's protocol identification thread captures the serial port data stream, extracts the first 32 bytes, and matches them sequentially with the protocol fingerprints in the feature library. The matching algorithm employs weighted multi-mode matching, comprehensively scoring features such as baud rate, start bit, data frame length, and checksum position. Upon successful matching, the edge node determines the device type as a Vaisala PTU310 weather station and the protocol as MODBUS RTU.

[0048] Subsequently, the edge node loads the corresponding protocol conversion rule from the feature library. This rule consists of three parts: The field mapping table is used to map MODBUS register addresses to internal standard data fields; Unit conversion factors, such as converting Kelvin to Celsius; Verification algorithm.

[0049] The edge node maintains a protocol conversion instance for each device in memory, enabling automatic identification and protocol conversion for all connected devices. The entire process takes approximately 47 seconds, significantly improving deployment efficiency.

[0050] Before the protocol conversion, the edge nodes authenticate the monitoring terminals. In this embodiment, each terminal has a pre-installed digital certificate, and the edge nodes verify the validity and signature of the certificate, rejecting unauthenticated terminals from accessing the network. As an alternative, a pre-shared key authentication method can also be used.

[0051] (ii) Data acquisition, lightweight coding, and compression ratio linkage Edge nodes poll each sensor at a frequency of 1Hz. For numerical data, lossless compression is performed using differential pulse code modulation (DPCM); for waveform data from the ice seismic monitoring instrument, lossy compression is performed using discrete cosine transform.

[0052] The compression ratio is dynamically adjusted based on the comprehensive link quality score monitored in real time. In this embodiment, the calculation of the comprehensive link quality score is described in step (III).

[0053] When the overall score is higher than the first threshold of 70, the numerical data compression ratio is 4:1, and the waveform data retains 128 coefficients. When the score is between the second threshold of 35 and the first threshold of 70, the compression ratio is adjusted to 8:1, and the waveform data retains 32 coefficients. When the score is lower than the second threshold of 35, only key fields of the numerical data are transmitted, and the waveform data is stopped from being transmitted and is stored locally in its entirety. The lower the score, the higher the compression ratio.

[0054] As a preferred implementation of this embodiment, the edge node further introduces the priority index P of the data to be transmitted and the current remaining storage capacity rate S. rem Fine-tuning of the compression ratio is performed. The priority index P is determined based on the monitoring terminal type, data sampling frequency, and event urgency, ranging from [0,1], with 1 representing the highest priority; the remaining storage capacity rate S... rem Defined as the ratio of the node's remaining non-volatile storage space to the total storage capacity. The real-time compression ratio Rc is calculated using the following formula: Rc=R base ·f(LQS)·g(P)·h(S rem ) in: R base Base compression ratio: 4:1 by default for numerical data, and 10:1 by default for image / acoustic data.

[0055] f(LQS) is the link quality correction factor: In this case, α=50 and β=5. The worse the link, the higher the compression ratio.

[0056] g(P) is the priority correction factor: g(P) = 1 + λ·(1-P), where λ = 0.6. The lower the priority, the higher the compression ratio.

[0057] h(S rem h(S) is the storage urgency correction factor. rem )=1+γ·(1-S rem Let γ = 0.5. The less remaining storage, the higher the compression ratio.

[0058] Boundary constraints on the compression ratio: Edge nodes recalculate R every T minutes (e.g., 5 minutes). c The subsequent compression process is dynamically adjusted.

[0059] (III) Link Status Monitoring and Comprehensive Scoring Each edge node independently runs a link monitoring thread, sending a heartbeat probe packet every 30 seconds via the Iridium satellite module, recording the sending and acknowledgment times, and calculating the round-trip time. Simultaneously, it reads parameters such as signal-to-noise ratio, received signal strength indication, and packet loss rate from the satellite module driver.

[0060] The length of the sliding time window can be configured as a fixed duration (e.g., 30 seconds) or a variable duration (e.g., dynamically adjusted according to link stability). In this embodiment, a fixed window of 10 sample sizes is used to perform median filtering on the above parameters to eliminate spike interference caused by the instantaneous and drastic fluctuations of the polar link. Then, the filtered parameters are weighted and fused to obtain the Link Quality Score (LQS) ranging from 0 to 100. The scoring formula is: LQS = 0.4 × SNR norm + 0.3×(1-packet loss rate) + 0.2×(1-RTT / 30s) + 0.1×RSSI norm , Among them, SNR norm The signal-to-noise ratio is a normalized value, and the packet loss rate is directly used as a decimal between 0 and 1. The RTT is normalized with a maximum of 30 seconds. The weighting coefficient is dynamically distributed by the remote data center based on historical statistics; as an alternative implementation, edge nodes can also adaptively adjust the weighting coefficient based on locally recorded historical transmission success rates.

[0061] The adaptive adjustment process of the weight coefficients can be as follows: if the historical transmission success rate of the edge node continues to increase to the preset success threshold, then the weight coefficient of (1-packet loss rate) is reduced and the other weight coefficients are increased; if the historical transmission success rate of the edge node continues to be lower than the preset minimum success threshold, then the weight coefficient of (1-packet loss rate) is increased and the other weight coefficients are reduced.

[0062] This embodiment uses data center distribution as an example. The preset transmission threshold is distributed by the data center and defaults to LQS ≥ 35. Edge nodes synchronize the threshold configuration every 10 minutes.

[0063] (iv) Adaptive transmission / storage switching and out-of-band deletion notification like Figure 3 As shown, when LQS ≥ 35, the edge node enters online transmission mode. It packages the compressed lightweight data packet, adds a packet sequence number, timestamp, and HMAC-SHA256 message authentication code, and submits it to the Iridium module for transmission after using SM4 symmetric encryption.

[0064] When LQS < 35, such as when a polar geomagnetic storm causes the SNR to drop to 1 dB, or when the Iridium module reports no service, the edge node automatically switches to local persistent storage mode. It writes lightweight data packets to a pre-allocated circular storage area in the eMMC, while maintaining a block index table to record the starting packet sequence number, time range, and priority flag.

[0065] When the remaining eMMC space falls below 10%, the edge node triggers cleanup: blocks are sorted by priority, with the lowest priority and earliest timestamp blocks being deleted first. Before deleting each block, the edge node extracts the digests of the first and last data packets in that block to obtain the packet sequence number and timestamp, packages them into a very short message of no more than 50 bytes, and sends it to the data center via an out-of-band channel independent of the main satellite link—Iridium SBD Short Data Service—informing the data loss area. The remote data center receives this and records the data gap for subsequent manual data collection.

[0066] This embodiment also sets a second threshold (e.g., 20). When the Link Quality Score (LQS) is lower than the preset transmission threshold of 35 but higher than the second threshold of 20, the edge node further determines whether the currently collected data belongs to the highest priority event. Highest priority events include: ice tremor events, extreme weather events, navigation collision warning signals, and equipment failure alarms. If it belongs to the highest priority event, the edge node attempts to transmit online by reducing the bit rate and increasing forward error correction coding; otherwise, it switches to local persistent storage mode.

[0067] (v) Fine-grained resume transmission after link recovery Once the link quality recovers to LQS ≥ 35 and remains stable for 3 minutes, the edge node enters resume mode. It first sends a resume request to the data center, carrying its local ID and the range of packet indexes stored during the network outage, such as packet sequence numbers 15000~17820. The data center queries the list of successfully received packet sequence numbers locally and returns a list of missing packets, such as 15005, 15012, 15089…

[0068] The edge node reads only the corresponding data packets from the eMMC based on the missing data list and transmits them, skipping packets that have already been acknowledged by the remote end. Simultaneously, the edge node maintains a priority queue: high-priority packets such as device fault alarms, ice shock events, and AIS collision warnings are sent first; low-priority packets such as routine meteorological data are sent subsequently; within the same priority, packets are sent in chronological order. Packets that fail to transmit after more than three retransmissions are marked as failed and logged.

[0069] (vi) Security verification of remote control commands When the data center sends remote control commands to edge nodes, it uses an SM2 private key to sign the command digest and sends the original command text, signature value, and timestamp together. Upon receiving the command, the edge node first verifies the timestamp discrepancy, then verifies the signature using a pre-stored SM2 public key. If the verification is successful, the command is executed; otherwise, the request is rejected, an abnormal access log is logged, and an alarm is sent. The execution result is encrypted and transmitted back.

[0070] (vii) Multi-edge node collaboration In this embodiment, three adjacent unmanned ice stations were also deployed, approximately 5 kilometers apart, with each station's edge nodes equipped with LoRa wireless modules. When the Iridium link of a node completely fails for more than 24 hours, that node broadcasts a proxy request via LoRa, carrying its own cached data volume. Neighboring nodes reply with their LQS and remaining storage capacity. A proxy node is selected based on preset rules: priority is given to nodes with LQS > 50 and remaining capacity > 200MB; if multiple nodes have LQS, the one with the highest LQS is selected.

[0071] The proxy node collects cached data indexes from other nodes and merges them for deduplication: if the timestamps of data packets uploaded by two nodes differ by less than ±1 second, and the numerical deviations for the same device ID and the same physical quantity are less than 5%, they are considered duplicate data, and only one copy is retained. The proxy node then sends the merged data index to the data center via its own Iridium link to initiate a resume transmission request. If a proxy node fails during transmission, the remaining nodes re-elect a new proxy and resume transmission from the point of interruption based on the records already received by the data center.

[0072] This embodiment also provides a lightweight edge data transmission system for monitoring terminals in polar environments, characterized in that it includes a memory and a processor. The memory stores a computer program, and the processor calls the computer program to execute the steps of the above-described lightweight edge data transmission method for monitoring terminals in polar environments.

[0073] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. An edge lightweight data transmission method for a polar environment-oriented observation monitoring terminal, characterized in that, Includes the following steps: At edge nodes in polar environments, the device type and communication protocol of the monitoring terminals connected to the polar edge nodes are identified by a pre-set multi-source heterogeneous device protocol feature library and adaptive matching algorithm, so as to call the corresponding protocol conversion rules and obtain the converted raw data. The system acquires the connectivity status of the current communication link of the edge node. If the link remains active, it collects the link quality parameters of the current communication link and performs parameter filtering and normalization using a fixed or variable duration sliding time window. Then, it performs weighted fusion to obtain a comprehensive link quality score. The weighting coefficients for the weighted fusion are dynamically distributed by the remote data center or adaptively adjusted by the edge node based on historical transmission success rates. The link quality parameters include at least one of the following: signal-to-noise ratio, received signal strength indication, packet loss rate, round-trip time, and available bandwidth. The raw data is compressed in real time to generate a lightweight data packet; The link quality comprehensive score is compared with a preset score threshold. If it is greater than or equal to the preset score threshold, the edge node will transmit the lightweight data packet back to the remote data center in real time; otherwise, it will be stored locally and the unacknowledged lightweight data packet will be transmitted to the remote data center after the current communication link is restored.

2. The edge lightweight data transmission method for a polar region environment facing monitoring terminal according to claim 1, characterized in that, The method further includes: The priority of the lightweight data packet is determined based on the device type of the monitoring terminal, the data sampling frequency, the urgency of the event, or the user's preset tags. When the overall link quality score is lower than a preset score threshold but higher than a preset second threshold, it is determined whether the priority of the lightweight data packet is the highest priority. The highest priority corresponds to events such as ice shock events, extreme weather events, navigation collision warning signals, and equipment failure alarms. If so, the edge node adopts a reduced bit rate, increases forward error correction coding, or adopts a modulation method with preset robustness enhancement for online transmission. Otherwise, it performs local storage. During the process of transmitting lightweight data packets to the remote data center, the edge node transmits higher-priority lightweight data packets first, followed by lower-priority lightweight data packets; for lightweight data packets of the same priority, they are sent according to the order in which the data-generating events occur.

3. The edge lightweight data transmission method for a polar region environment facing monitoring terminal according to claim 1, characterized in that, The method further includes: When multiple edge nodes cover the same observation area, each edge node exchanges its own link quality comprehensive score and remaining storage capacity information through short-range wireless communication, and selects the node with the highest link quality comprehensive score and remaining storage capacity greater than the preset capacity threshold as the proxy node. The proxy node collects the indexes of untransmitted lightweight data packets stored locally by other edge nodes within the observation area, merges and deduplicates them, and then sends a unified retransmission request to the remote data center. The merging and deduplication process includes: if the difference in timestamps of data packets uploaded by two edge nodes is less than a preset time window and the numerical deviation of the same physical quantity corresponding to the same monitoring terminal is less than a preset percentage, then the data is determined to be duplicated and only one copy is retained. If a proxy node fails during transmission, the remaining edge nodes within the observation area will re-elect a new proxy node through an election mechanism and resume transmission from the point of interruption based on the records already received by the remote data center.

4. The edge lightweight data transmission method for a polar region environment facing monitoring terminal according to claim 1, characterized in that, The protocol conversion rules include a field mapping table, unit conversion coefficients, and verification algorithms, which are used to uniformly convert heterogeneous data frames output by monitoring terminals from different manufacturers or types into the internal standard format of the edge nodes. The field mapping table is used to record the correspondence between source protocol fields and standard format fields; The unit conversion factor is used to uniformly convert physical quantities with different dimensions; The verification algorithm includes cyclic redundancy check.

5. The edge lightweight data transmission method for a polar region environment-oriented monitoring terminal according to claim 1, characterized in that, The real-time compression process of the raw data includes: The numerical observation data in the original data are compressed using a lossless compression method based on differential pulse code modulation or run-length encoding. A lossy compression method based on discrete cosine transform is used to compress the image and acoustic observation data in the original data; The dynamic adjustment process of the compression ratio in the real-time compression process includes: finely adjusting the pre-set base compression ratio based on the comprehensive link quality score, the priority index of the original data, and the current remaining storage capacity of the edge nodes to obtain the dynamically adjusted compression ratio, and applying boundary constraints. The dynamic adjustment calculation expression for the compression ratio is as follows: Rc = R base • f(LQS) • g(P) • h(S rem ) g(P) = 1 + λ·(1-P) h(S rem )=1+γ·(1-S rem ) In the formula, Rc R is the dynamically adjusted compression ratio. base The base compression ratio is f(LQS), where f(LQS) is the link quality correction factor. LQS For the comprehensive link quality score, α and β are preset parameters, g(P) is the priority correction factor, λ is the priority scaling parameter, P is the numerical result of the priority, and h(S) is the priority value. rem S is the storage urgency correction factor. rem This represents the current remaining storage capacity rate; The calculation expression for the boundary constraints is: wherein Rmax is the maximum value of the compression ratio, Rmin is the minimum value of the compression ratio.

6. The edge lightweight data transmission method for a polar region environment facing monitoring terminal according to claim 1, characterized in that, The method also includes a dynamic adjustment process for the scoring threshold, which is as follows: The remote data center dynamically issues scoring thresholds based on the historical link quality comprehensive score of the current polar environment; Alternatively, the scoring threshold can be gradually lowered after a preset number of consecutive successful transmissions at the edge node, and gradually raised after a preset number of consecutive failed transmissions at the edge node.

7. The edge lightweight data transmission method for a polar region environment facing monitoring terminal according to claim 1, characterized in that, The method stores lightweight data packets locally in the non-volatile storage of the edge node for local storage; if the storage space of the edge node is insufficient, the lightweight data packet with the lowest priority and the earliest generation time is deleted first, and the remote data center is notified through the out-of-band channel. The out-of-band channel is a short message service, a satellite beacon signal, or a low-frequency auxiliary communication channel; The notification to the remote data center via out-of-band channel includes the timestamp range of the deleted lightweight data packets, the packet sequence number range, and the identification of the monitoring terminal to which the data belongs.

8. The edge lightweight data transmission method for a polar region environment facing monitoring terminal according to claim 1, characterized in that, The process of transmitting lightweight data packets that were not successfully acknowledged to a remote data center includes: The edge node sends a list of data packet indexes or timestamp ranges stored during the network outage to the remote data center, receives missing data packet identifiers from the remote data center, and reads the corresponding data packets from local storage based on the missing data packet identifiers for transmission. After the current communication link is restored, the method counts the duration of stable link quality. After waiting for the stable link quality to reach a preset stable duration threshold, it then transmits the lightweight data packets that were not successfully acknowledged to the remote data center. If a single lightweight data packet fails to be retransmitted multiple times, it is marked as a transmission anomaly and logged.

9. The edge lightweight data transmission method for a polar region environment facing monitoring terminal according to claim 1, characterized in that, The method also includes security protection steps: The system authenticates the access monitoring terminals to reject unauthorized access; before storing and transmitting lightweight data packets, it encrypts them using a symmetric encryption algorithm and attaches a hash message authentication code; it uses an asymmetric algorithm to sign and verify the received remote control commands, executes the commands after successful verification and encrypts and sends the execution result back to the command sender, and records an exception log and triggers an alarm if the verification fails. The encryption key of the symmetric encryption algorithm is updated periodically, either through out-of-band channel distribution or through dynamic generation based on time synchronization.

10. An edge lightweight data transmission system for a polar environment facing observation monitoring terminal, characterized in that, It includes a memory and a processor, the memory storing a computer program, the processor invoking the computer program to perform the steps of the method as described in any one of claims 1 to 9.

Images