A network interruption adaptive edge iot gateway data breakpoint continuation system and method
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-19
- Publication Date
- 2026-06-23
Smart Images
Figure CN122268761A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of network optimization technology, and more specifically, to an edge IoT gateway data interruption resume system and method with network interruption adaptive capability. Background Technology
[0002] With the rapid development of IoT technology and its widespread application in smart manufacturing, smart cities, and smart homes, the number of IoT devices is growing exponentially, significantly increasing the complexity of IoT networks. In typical IoT application scenarios, numerous devices interact with the cloud through edge IoT gateways to achieve real-time monitoring, remote control, and data analysis. However, because IoT networks typically operate in highly heterogeneous and dynamic environments, network outages or fluctuations have become significant issues affecting system stability and data integrity. For example, network outages can be caused by low-quality links, network congestion, or device failures, leading not only to data loss but also potentially impacting the real-time requirements of critical business operations. Therefore, ensuring reliable data transmission and integrity during network outages or fluctuations has become a crucial research topic in the current IoT technology field.
[0003] Existing IoT network optimization technologies mainly focus on three aspects: network status perception and prediction, data prioritization, and data transmission caching and recovery. However, these technologies still have shortcomings in practical applications. First, in terms of network status perception, traditional methods typically rely on a single parameter (such as network latency or packet loss rate) for monitoring, which cannot comprehensively reflect the complex changes in network status, especially when multiple parameters interact, making it difficult to achieve high-precision network fluctuation prediction. Second, in terms of data prioritization, existing methods are mostly based on fixed weights or static rules for data classification, lacking dynamic adjustment capabilities and unable to adapt to complex and ever-changing business needs or real-time requirements. Finally, in the field of data transmission caching and recovery, existing caching mechanisms typically employ fixed layering or simple compression techniques, making it difficult to dynamically adjust caching strategies according to network conditions and data characteristics, resulting in low storage resource utilization. In addition, current breakpoint resumption methods mostly rely on fixed transmission rates or simple retransmission mechanisms, failing to fully consider real-time changes in network bandwidth, resulting in limited transmission efficiency and data completion capabilities. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention is proposed. This invention provides a network interruption adaptive edge IoT gateway data breakpoint resumption system and method, which can, to some extent, solve the problems of difficulty in fully sensing and predicting network state changes, inability to dynamically adapt data priorities, and the lack of flexibility and resource optimization capabilities in caching and transmission mechanisms.
[0005] According to one aspect of the present invention, a method for resuming interrupted data transmission at the edge IoT gateway with network interruption adaptation is provided, comprising: Deploy a multimodal network awareness engine, combine network latency, packet loss rate and link quality index to calculate network health score and achieve predictive identification of network status; Construct a data hierarchy matrix, extract data features, dynamically adjust feature weights based on reinforcement learning algorithms, and generate data priority scores. When a network anomaly warning is detected, a hierarchical elastic caching mechanism is used to intelligently group newly generated data according to priority and relevance and store it in a hierarchical manner. After the network is restored, the adaptive bandwidth-aware tiered resume module allocates bandwidth resources based on the current network capacity and hierarchical cache priority, adjusts the resume rate, and completes data.
[0006] Furthermore, the multimodal network perception engine includes a network parameter acquisition module, a state fusion calculation module, and a prediction and early warning module.
[0007] Furthermore, the prediction and early warning module is designed with an ultra-lightweight heartbeat frame mechanism based on the improved MQTT protocol. It combines round-trip delay jitter mode and loss situation to construct a network fluctuation prediction model and uses a long short-term memory network to output the probability distribution of network fluctuations in the next 800 milliseconds.
[0008] Furthermore, the data grading matrix extracts four features of the data: timeliness, criticality, business relevance, and security level. Based on a reinforcement learning algorithm, the feature weights are dynamically adjusted to generate a data priority score. Based on the priority scores, the data is divided into emergency alarm data, real-time control data, and non-real-time statistical data.
[0009] Furthermore, the tiered elastic caching mechanism includes three levels of caching hierarchy: The first-level cache is a high-speed memory buffer that prioritizes storing emergency alarm data. The second-level cache is the main memory storage area, which stores real-time control data and highly relevant non-real-time statistical data. The third-level cache is a persistent storage area that stores low-priority, non-real-time statistical data.
[0010] Furthermore, the allocation coefficient of the hierarchical elastic caching mechanism Determined by the Network Volatility Severity Index (NTI), the formula is as follows: ,in, This is a network fluctuation severity index.
[0011] Furthermore, the network fluctuation severity index The formula is expressed as: ,in, The standard deviation of the round-trip time of a heartbeat frame. This represents the average round-trip time of a heartbeat frame. Heartbeat frame loss rate, The rate of change of round-trip time, This is the normalized value of the network health score. These are the weighting coefficients.
[0012] Furthermore, based on the aforementioned three-level caching hierarchy, in order to organize cached data more efficiently, an improved weighted cosine similarity algorithm is used to calculate the correlation coefficient between data. The formula is expressed as: ,in, and Representing data respectively and data In the Values on dimensional features, For the first Weights of dimensional features Indicates the first Information gain of dimensional features This represents the total number of feature dimensions.
[0013] Furthermore, the hierarchical transmission rule of the resume module is as follows: Emergency alarm data in the first-level cache uses a single-packet confirmation mechanism; The real-time control data in the second-level cache uses a dynamically adjusted sliding window mechanism. Non-real-time statistics in the third-level cache are transmitted in batch mode.
[0014] According to another aspect of the present invention, an edge IoT gateway data interruption resume system with network interruption adaptation is provided, comprising: The network awareness module is used to calculate the network health score and achieve predictive identification of network status by combining network latency, packet loss rate and link quality index. The data classification module is used to extract data features, dynamically adjust feature weights based on reinforcement learning algorithms, and generate data priority scores. The hierarchical caching module is used to intelligently group and store newly generated data in a hierarchical elastic caching mechanism according to priority and relevance when a network anomaly warning is detected. The tiered resume module is used to allocate bandwidth resources based on the current network capacity and hierarchical cache priority, adjust the resume rate, and complete data.
[0015] Compared with existing technologies, this invention, by deploying a multimodal network perception engine, achieves comprehensive perception and prediction of network latency, packet loss rate, and link quality index, effectively improving the accuracy of network status monitoring; it constructs a data hierarchy matrix and dynamically adjusts data feature weights based on reinforcement learning algorithms, realizing intelligent allocation of data priorities and significantly improving the transmission efficiency of critical data; it adopts a hierarchical elastic caching mechanism, dynamically adjusting the caching strategy according to network fluctuations and data characteristics, optimizing the utilization of storage resources; and through an adaptive bandwidth-aware tiered resume module, it allocates bandwidth and adjusts the resume rate according to the current network capacity, achieving efficient data completion and ensuring transmission reliability. This significantly improves the reliability, real-time performance, and resource utilization efficiency of data transmission. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart illustrating a network interruption adaptive edge IoT gateway data breakpoint resumption method according to an embodiment of the present invention. Detailed Implementation
[0017] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.
[0018] As mentioned in the background section, existing technologies suffer from three main problems: First, traditional network status awareness technologies typically rely on a single parameter (such as latency or packet loss rate), which cannot comprehensively and accurately reflect the complex changes in network status, especially under the influence of multiple parameters, making it difficult to achieve high-precision network fluctuation prediction. Second, existing data priority allocation methods mostly use fixed weights or static rules, which cannot be dynamically adjusted according to changes in business scenarios and data characteristics, resulting in insufficient transmission efficiency of critical data. Third, existing breakpoint resume methods and caching mechanisms lack flexibility, making it difficult to dynamically adjust the transmission rate or caching strategy according to network status and data traffic, resulting in low resource utilization and low transmission efficiency.
[0019] Figure 1 This is a system block diagram of a network interruption adaptive edge IoT gateway data breakpoint resumption method according to an embodiment of the present invention. Figure 1 As shown, a method for resuming interrupted data transmission at the edge IoT gateway in an adaptive manner to network interruption includes: S1: Deploy a multimodal network awareness engine that integrates network latency, packet loss rate, and link quality index to calculate network health score and achieve predictive identification of network status.
[0020] Deploy a multimodal network perception engine, which includes a network parameter acquisition module, a state fusion calculation module, and a prediction and early warning module; The network parameter acquisition module synchronously collects network latency, packet loss rate, and link quality index. The network latency is obtained by the round-trip time of active probe packets, the packet loss rate is calculated by the reception of consecutive sequence numbered data packets, and the link quality index is measured by the signal strength to noise ratio. The state fusion calculation module uses a sliding time window of 200 milliseconds. First, network latency is normalized: a latency less than 50 milliseconds is assigned 100 points, a latency greater than 500 milliseconds is assigned 0 points, and linear interpolation is used to calculate the score when the latency is between 50 and 500 milliseconds. For packet loss rate, a 0% loss rate is assigned 100 points, a loss rate of 20% or higher is assigned 0 points, and the score decreases linearly proportionally when the loss rate is between 0% and 20%. For link quality... The index is assigned a score of 100 when the signal-to-noise ratio (SNR) is greater than 40 dB, and a score of 0 when the SNR is less than 10 dB. When the SNR is between 10 dB and 40 dB, a piecewise function mapping is used for scoring, with a steeper linear mapping used in the 10-20 dB range to reflect the significant impact of low SNR on communication quality. The scores of the above three items are weighted and summed in a weighted ratio of 4:3:3 to obtain a network health score in the range of 0-100. A network health score below 65 is considered an abnormal network condition, and a network health score below 40 is considered a network outage.
[0021] The prediction and early warning module is based on an improvement to the standard MQTT protocol, which designs an ultra-lightweight heartbeat frame mechanism. The ultra-lightweight heartbeat frame simplifies the MQTT protocol's PINGREQ and PINGRESP messages, retaining only a 2-byte fixed frame header and a 4-byte high-precision timestamp, reducing the original MQTT heartbeat packet from a minimum of 12 bytes to 6 bytes, thus reducing network load. The edge IoT gateway and the cloud transmit ultra-lightweight heartbeat frames bidirectionally at 15-millisecond intervals. When the gateway sends an ultra-lightweight heartbeat frame, it records the timestamp of the sending time. When it receives an ultra-lightweight heartbeat frame response from the cloud, it calculates the round-trip time and records the timestamp of the receiving time. If no response to an ultra-lightweight heartbeat frame is received for three consecutive times, the state is marked as heartbeat loss.
[0022] Furthermore, a network fluctuation prediction model is constructed by analyzing the jitter patterns and loss patterns of ultralight heartbeat frames. The jitter pattern refers to the variance of the round-trip latency of multiple consecutive ultralight heartbeat frames, while the loss pattern refers to the loss rate and temporal distribution characteristics of ultralight heartbeat frames within a specific time window. Specifically, the prediction and early warning module first calculates the round-trip latency of each ultralight heartbeat frame and maintains a sliding queue of latency values of length 20. The range of values in this queue that is three times the standard deviation is used as the normal jitter range. When the round-trip latency of an ultralight heartbeat frame exceeds the normal jitter range, it is recorded as a jitter anomaly. Simultaneously, the prediction and early warning module maintains a 1-second heartbeat loss recording window, recording the time of loss and the number of consecutive losses of ultralight heartbeat frames. When more than three jitter anomalies occur and the latency of these anomalies shows an increasing trend, or when the loss pattern within the heartbeat loss recording window exhibits a specific periodicity or concentration, a network fluctuation early warning prediction is triggered.
[0023] The network fluctuation prediction model employs a lightweight long short-term memory network structure. It takes the jitter pattern feature vector and the loss situation feature vector within the past 500 milliseconds as input and outputs the probability distribution of network fluctuations within the next 800 milliseconds. When the probability distribution exceeds 75% at any time point, a network fluctuation warning is generated. If the peak value of the predicted network fluctuation probability distribution exceeds 90%, a severe network fluctuation warning is generated.
[0024] The network fluctuation prediction model can issue an early warning signal 300-500 milliseconds before the actual deterioration of the network status. The early warning signal is divided into three levels, corresponding to mild fluctuation, moderate fluctuation and severe fluctuation, respectively, providing a basis for subsequent data caching strategy adjustments.
[0025] S2: Construct an intelligent data hierarchical matrix, extract data features, dynamically adjust feature weights based on reinforcement learning algorithms, and generate data priority scores.
[0026] Constructing an intelligent data hierarchy matrix involves dynamically prioritizing data by establishing a multi-dimensional evaluation framework. The construction process begins by designing the basic structure of the matrix, creating an n×m two-dimensional matrix where n represents the data type dimension (such as sensor data, control commands, system logs, etc.) and m represents the evaluation feature dimension (timeliness, criticality, business relevance, and security level). Initial weight values are then assigned to each cell of the matrix.
[0027] Based on the constructed intelligent data hierarchy matrix, it is deployed on the edge IoT gateway to perform multi-dimensional feature analysis on the data collected by the edge IoT gateway and achieve adaptive classification of data priorities.
[0028] Specifically, the intelligent data grading matrix first extracts four core features of the data: timeliness, criticality, business relevance, and security level, and then assigns weight coefficients to each feature: Among them, the timeliness feature is scored by analyzing the effective time window of the data. When the effective time window of the data is less than 100 milliseconds, a full score of 100 points is given. When the effective time window of the data is greater than 10 minutes, a minimum score of 10 points is given. When the effective time window of the data is between 100 milliseconds and 10 minutes, the score is calculated using a logarithmic decay function. Key features are scored by assessing the impact of data on system functionality and security. A full score of 100 points is given when the data is directly related to the system’s core control functions or security mechanisms, and a minimum score of 20 points is given when the data is only used for non-critical business statistical analysis. The business relevance feature determines the score by calculating the correlation between the data and the current active business process. The semantic similarity algorithm is used to match the data content with the keywords of the active business scenario. A full score of 100 points is given when the matching degree exceeds 90%, and a minimum score of 30 points is given when the matching degree is less than 30%. Security level features are scored based on the sensitivity and confidentiality requirements of the data. Data containing personal privacy, key parameters, or authentication information is given a full score of 100 points, while data that is completely public and contains no sensitive content is given a minimum score of 10 points.
[0029] The intelligent data grading matrix adopts an adaptive weight allocation mechanism based on specific application scenarios. In the initial state, the weight coefficients of the four features are timeliness 0.35, criticality 0.3, business relevance 0.2, and security level 0.15, respectively. When specific types of network anomalies or changes in business patterns are detected during system operation, the weights of each feature are dynamically adjusted through reinforcement learning algorithms. For example, when the system experiences frequent network interruptions, the weight of the timeliness feature is increased to 0.45-0.5 to prioritize the transmission of time-sensitive data. When the system is in a high-security scenario, the weight of the security level feature is increased to 0.3-0.4 to ensure priority protection of sensitive data. The weight adjustment adopts a smooth transition mechanism to avoid drastic fluctuations in priority strategies.
[0030] When the weighted feature score of the data exceeds 85, it is classified as emergency alarm data; when the weighted feature score of the data is between 60 and 85, it is classified as real-time control data; and when the weighted feature score of the data is below 60, it is classified as non-real-time statistical data.
[0031] Among them, the reinforcement learning algorithm is the Q-learning reinforcement algorithm. This algorithm takes the current network state, business mode and data transmission result as the environment state, the adjustment range of the four feature weights as the action space, and the weighted combination of data transmission success rate, business continuity index and resource utilization efficiency as the reward function. In practice, the system assesses the environmental state every 30 seconds, calculates the optimal action based on the reward value of the previous period, and performs weight adjustments. The initial exploration rate is set to 0.3, which is gradually reduced to 0.05 as the system runs to balance exploration and utilization. When a sudden drop in network bandwidth exceeding 30% is detected, the algorithm immediately triggers a special state assessment, increasing the adjustment step size of the timeliness feature weights to 0.05-0.08 to quickly respond to network changes. When the business mode switches from routine monitoring to emergency response, the system identifies this state transition and simultaneously enhances the criticality and timeliness features. The weights of business relevance features are reduced; to prevent the system from becoming unstable due to overly drastic weight adjustments, a weight change rate limit is set, with a single adjustment not exceeding 15% of the current value, and the sum of all weights always remains 1; an experience replay mechanism is adopted, storing samples of the most recent 100 state transitions, and randomly selecting 32 samples for batch updates during each training session to accelerate algorithm convergence and improve generalization ability; at the same time, when there is no significant improvement in transmission performance after 5 consecutive weight adjustments, a learning rate decay mechanism is triggered to reduce the aggressiveness of weight adjustments and find a more stable weight configuration.
[0032] Furthermore, an enhanced time-series label is generated for each data point based on its priority. The enhanced time-series label includes a 16-bit data source identifier, a 32-bit precise millisecond-level timestamp, a 16-bit priority code, and a 32-bit Reed-Solomon self-healing code.
[0033] The 16-bit data source identifier consists of four bits: the first four bits represent the device category, the middle eight bits represent the specific device ID, and the last four bits represent the data acquisition channel, ensuring unique identification of the data source in complex IoT networks. The 32-bit precise millisecond-level timestamp consists of four bits: the first twelve bits represent the number of days since the standard reference time, the middle ten bits represent the number of seconds within a day, and the last ten bits represent the millisecond count, supporting a time span of up to 136 years and millisecond-level precision. The 16-bit priority encoding consists of four bits: the first four bits represent the primary priority category of the data (emergency alarm, real-time control, or non-real-time statistics), the middle four bits represent the sub-priority of the data within its category, and the last eight bits store the data's timeliness decay curve parameters for subsequent transmission decisions. The 32-bit Reed-Solomon self-healing encoding can automatically recover complete information when the time-series tag is partially damaged, providing up to 30% data repair capability.
[0034] Meanwhile, the edge IoT gateway calculates a collision-resistant hash check code for each piece of data. The collision-resistant hash check code uses an improved SHA-256 algorithm, which significantly reduces the probability of hash collisions by introducing data feature entropy as a salting parameter. Specifically, the improved algorithm first calculates the entropy characteristics of the original data, segments the data using a sliding window, and statistically analyzes the frequency and distribution pattern of each byte to generate a 16-byte data feature entropy value. Then, it performs an XOR operation between the data feature entropy value and the time-series label of the original data to obtain customized salting parameters. Next, it modifies the initial hash value of the standard SHA-256 algorithm, replacing the fixed constant with a dynamic initial value mixed with the salting parameters. In the message expansion stage, it introduces an adaptive round function based on data type. When processing emergency alarm data, a non-linear transformation is used to enhance the hashing effect; when processing non-real-time statistical data, a round function with lower computational complexity is used to improve efficiency. During the compression function calculation, an additional obfuscation step with adaptive data length is introduced. For data blocks larger than 1KB, an XOR operation is inserted between the intermediate hash value and a portion of the original data block after processing every 64 bytes to further enhance the avalanche effect. Finally, it outputs a 256-bit hash value, but instead of using this value directly as a checksum, it is interleaved with the enhanced time-series label of the data in a specific pattern to generate the final collision-resistant hash checksum.
[0035] Furthermore, a distributed transmission state graph is constructed. By updating node states in real time, the complete topology and execution status of data transmission are maintained. Specifically: First, a memory-based directed acyclic graph data structure is initialized upon edge gateway startup. Each node represents a piece of data to be transmitted, with its status field initially set to "Pending Send". When a piece of data begins transmission, the corresponding node's status is updated to "Transmitting", and the transmission start timestamp and current retries are recorded. When a data transmission confirmation message is received, the transmission integrity is verified by comparing a collision-resistant hash checksum. If the verification is successful, the node's status is updated to "Completed", triggering a status check of all dependent nodes of that node. When all of a node's preceding dependent nodes are in the "Completed" state, its status is updated to "Ready". If data transmission times out or a transmission failure notification is received, the node... Points are marked as "failed" and the retry interval is determined based on data priority and retry strategy. Emergency alarm data uses an exponential backoff algorithm for rapid retry, while non-real-time statistical data uses a linearly increasing retry interval. When a change in network conditions is detected, the distributed transmission state graph performs a global state scan, recalculating the transmission priority for all nodes in the "pending transmission" and "failed" states. After network recovery, data with high priority that has not been successfully transmitted for a long time is processed first. If a node still fails to transmit after reaching a threshold number of retries, the system will perform cascading impact analysis to identify all data that directly and indirectly depends on the failed node and update its status to "dependency not satisfied" to avoid invalid transmission attempts. The distributed transmission state graph uses an optimistic locking mechanism to update the node state. Before each state change, the node version number is checked, and the update is only performed and the version number is incremented if the versions match, thus resolving concurrent update conflicts. In the scenario of edge gateway cluster deployment, the gateways periodically synchronize the differences in the state graph through the Gossip protocol. Each gateway selects 1-3 neighboring gateways per second to exchange state summaries. After detecting differences, only the changed node information is transmitted, ensuring state consistency while minimizing synchronization overhead.
[0036] S3: When a network anomaly warning is detected, a hierarchical elastic caching mechanism is adopted to intelligently group newly generated data according to priority and relevance and store it in a hierarchical manner.
[0037] When a network anomaly warning is detected, the edge IoT gateway activates a hierarchical elastic caching mechanism, which consists of three levels of caching to achieve differentiated data storage and dynamic expansion.
[0038] Specifically, upon receiving an early warning signal from the network fluctuation prediction model, the hierarchical elastic caching mechanism immediately allocates additional computing resources for data caching processing. The resource allocation ratio is determined by the network fluctuation severity index. The decision, expressed by the formula, is as follows: ,in, The standard deviation of the round-trip time of a heartbeat frame. This represents the average round-trip time of a heartbeat frame. Heartbeat frame loss rate, The rate of change of round-trip time, This is the normalized value of the network health score. These are the weighting coefficients.
[0039] Meanwhile, the network volatility severity index The resource dynamic allocation coefficient directly affects the tiered elastic caching mechanism. The formula is expressed as: This coefficient This determines the additional resource multiplier that the system allocates to the caching mechanism, with a maximum of 5 times the base resource.
[0040] at the same time, This not only affects the overall resource allocation but also directly determines the space allocation ratio among the three levels of cache. The first level cache is a high-speed memory buffer that uses a non-blocking circular queue structure, prioritizing the storage of urgent alarm data and allocating a certain proportion of the total cache space. Determined by the following formula: , The second-level cache is the main memory storage area, using a partitioned hash table structure. It primarily stores real-time control data and highly correlated non-real-time statistical data, and allocates a certain proportion of the total cache space accordingly. Determined by the following formula: , The third-level cache is a persistent storage area, implemented using flash memory or solid-state drives, storing other non-real-time statistical data and historical overflow data. The proportion of the total cache space allocated to it, S3S_3S3, is ensured to be 1 by a balance equation: , With the severity of network fluctuations The ratio of Level 1 to Level 2 cache is dynamically increased to cope with the caching needs of more urgent data. Specifically: If the proportion of emergency alarm data in the operating environment reaches or exceeds 25% of the total data flow, it can be controlled through the system configuration interface. The base value was increased from 0.15 to 0.25, while the scaling factor in the tanh function was adjusted from 2 to 2.5, and the offset was adjusted from -1 to -0.8, resulting in a steeper growth slope for the dynamic response curve under moderate network fluctuations. When the deployment scenario requires control command response latency to be strictly below 50 milliseconds, the base value can be increased. The base value was increased from 0.35 to 0.45, and the coefficient of the sine function was increased from 0.1 to 0.15, allowing the second-level cache to scale up to 60% of the total cache space under extreme network conditions; if the system is monitored for more than 7 consecutive days, The index average has consistently been above 0.65, and can be adjusted. Packet loss rate in the calculation formula The weight was increased from 0.35 to 0.45, while... rate of change The weight of [item name] is increased from 0.2 to 0.25, and the weights of other items are reduced accordingly; in the total [item name]... On resource-constrained edge devices with less than 256MB, the following should also be included: and The base values are reduced to 0.12 and 0.28 respectively, the coefficient of the dynamic adjustment item is halved, and at least 60% of the space allocation is reserved for the third-level flash-based persistent cache. All parameter adjustments can be configured and take effect within 60 seconds through the management console. The system will continuously monitor the performance indicators for 96 hours after the adjustment. When the cache hit rate drops by more than 5 percentage points or the average data processing latency increases by more than 15%, optimization suggestions will be automatically generated and rollback options will be provided.
[0041] Furthermore, after resource allocation is completed, the edge IoT gateway will intelligently group newly generated data according to priority and relevance. First, it will determine the basic priority value of the data based on the hierarchical results of the intelligent data hierarchical matrix. Then, the contextual importance of the data is calculated based on its temporal position in the business process. The final priority score is synthesized using the following formula. : ,in, The timeliness score represents the rate at which the value of data decays over time. These are the weighting coefficients.
[0042] Final priority score Directly determines the storage location of data in the L3 cache, high The data of the value is Values greater than 0.8 are allocated to the first-level cache, and medium values are allocated to the second-level cache. The value is 0.5 < Data values <0.8 are allocated to the second-level cache, low... Value Data values less than 0.5 are allocated to the third-level cache.
[0043] Meanwhile, to organize cached data more efficiently, an improved weighted cosine similarity algorithm is used to calculate the correlation coefficient between data. The formula is expressed as: ,in, and Representing data respectively and data In the Values on dimensional features, For the first Weights of dimensional features Indicates the first Information gain of dimensional features This represents the total number of feature dimensions.
[0044] When the correlation strength When the data exceeds the preset threshold, and data Data is grouped into the same group to ensure consistency during storage and subsequent transmission; based on the data value and the associated grouping results, each group of data is encrypted and stored using a layered encryption strategy with varying encryption strength. Priority score They are inversely proportional, as expressed by the formula: ,in, This is the lowest level of encryption applied to all data, representing the basic encryption strength. This is the maximum additional strength increment, which can be adjusted according to priority. To adjust the coefficients, the nonlinear relationship between priority and encryption strength is controlled.
[0045] Over time, cache utilization rate continuously rising, when When the preset threshold is exceeded or the network outage is expected to last longer than 30 minutes, the data semantic compression algorithm needs to be activated to expand the effective cache capacity; compression ratio target Based on current cache usage Dynamic calculation, expressed by the formula: ,in, This represents the current cache usage rate. The threshold for safe cache usage. For minimum compression ratio, This represents the maximum allowable compression ratio.
[0046] Computational Macro-level indicators for determining data compression strategies A higher value indicates a need for more aggressive compression strategies to cope with severe storage pressure.
[0047] Specifically, when the compression ratio target is determined Subsequently, for the time series data, adaptive sampling techniques are used for compression, with sampling intervals... By signal change rate Dynamic adjustment, expressed by the formula: , in, The basic sampling interval is the standard sampling frequency under normal circumstances. The standardized rate of change of the signal value per unit time. These are control parameters.
[0048] For structured data, this is achieved by calculating field importance scores. The importance score is calculated using the following formula: This determines the processing strategy for this field during the compression process. , in, For the first The normalized entropy values of each field. Score the relevance of business operations. Rate historical visit frequency. These are the weighting coefficients.
[0049] Field Importance Scoring This directly determines the processing strategy for the field during compression. Fields with scores above the threshold are retained in their entirety, while fields with lower scores may be compressed or temporarily discarded.
[0050] It is important to note the accuracy of data restoration after compression. Compared to the actual compression ratio There is a trade-off: , in, The actual compression ratio achieved. For reference compression ratio, The degradation coefficient for a specific data type.
[0051] By balancing these factors, we can estimate the impact of different compression strategies on data quality and select the optimal balance point.
[0052] After compression, storage space can be saved by an average of 5 to 20 times while preserving data value density to the maximum extent, and the effective cache capacity can be increased. The formula is expressed as: , in, Based on the basic physical cache capacity, For different types of data Compression ratio function, This is the data type distribution density function.
[0053] In addition, to evaluate the quality of compression, the overall information retention rate is calculated. : , in, For the first The retention ratio of each key information dimension For the corresponding importance weight, This represents the total number of key information dimensions.
[0054] Set a lower limit for information retention rate. This lower limit is related to the security criticality of the application. The relevant formula is expressed as follows: , in, Score the application's security criticality. This is the scaling parameter.
[0055] For applications with high security criticality, the lower limit of information retention rate is closer to 1, ensuring that core information is not lost.
[0056] As time goes on, the data in the cache needs to be managed for its lifecycle, which involves calculating the retention priority of each piece of data. The formula is expressed as: ,in, For the final priority score, This is a variable indicating successful transmission (1 for successful transmission, 0 otherwise). Indicates the elapsed time since the data was generated. This is the time-dependent attenuation coefficient.
[0057] Preservation Priority This determines the data's position in the competition for cache space, low High-value data is prioritized for cleanup when the cache approaches its capacity limit. The data is retained long-term to ensure that the system can still save the most critical business data in the event of a prolonged network outage.
[0058] S4: After the network is restored, bandwidth resources are allocated based on the current network capacity and hierarchical cache priority, the resume rate is adjusted, and data completion is achieved.
[0059] Once the network is restored, the edge IoT gateway immediately initiates an adaptive bandwidth-aware, tiered resumption strategy. Employing a precise network capacity assessment mechanism, it continuously sends ten test data packets with precise timestamps, each sent at 20-millisecond intervals. Upon receiving the test packets, the cloud immediately returns an acknowledgment packet. The edge calculates the round-trip time by measuring the time difference between sending and receiving acknowledgments, and simultaneously counts the number of packets that did not receive a response, thus calculating the current network packet loss. Based on these measurements, an estimate of the currently available network bandwidth is calculated, and the initial transmission rate is set to 60% of this estimate, reserving sufficient margin to prevent network congestion.
[0060] Available bandwidth is allocated according to cache importance: Level 1 cache receives 50% of the bandwidth; Level 2 cache receives 30%; and Level 3 cache receives the remaining 20%. The system reassesses network conditions every five seconds. When network stability is detected (round-trip time fluctuations do not exceed ±15% across six consecutive measurements, and packet loss remains below 2%), the total transmission rate is increased by 10% every twenty seconds until 85% of the estimated bandwidth is reached. If any measurement reveals a decline in network quality (round-trip time suddenly increases by more than 25% or packet loss exceeds 5%), the transmission rate is reduced to 70% of the current value, and any rate increase attempts are suspended for the next two minutes to prevent transmission instability caused by network fluctuations.
[0061] Furthermore, a finely differentiated transmission protocol is implemented for data at different cache levels. For high-priority data in the first-level cache, a single-packet acknowledgment mechanism is used, with each data packet size strictly controlled to within 1200 bytes. An acknowledgment must be received after transmission before the next packet is sent, ensuring reliable data transmission. If an acknowledgment is not received after 1.5 times the average round-trip time, the system will immediately retransmit the packet, attempting a maximum of three times. If retransmission fails after three attempts, the data is marked and placed back in the transmission queue, with a 30-second delay before retransmission is attempted again to avoid continuously consuming bandwidth resources.
[0062] For medium-priority data in the second-level cache, a dynamically adjusted sliding window mechanism is used. The initial window size is calculated based on the current network round-trip time and bandwidth, with a minimum range of 4,000 bytes and a maximum of 128,000 bytes. After four consecutive successful acknowledgments, the window size is increased appropriately; after each packet loss is detected, the window size is reduced to 75% of the current value, achieving adaptive traffic control.
[0063] For low-priority data in the third-level cache, a batch transmission mode is used, packaging multiple data packets into batches of up to 128,000 bytes for transmission. The interval between batches is dynamically adjusted based on the current network congestion, initially set at 50 milliseconds, but can fluctuate between 20 and 200 milliseconds. This significantly reduces protocol handshake overhead and improves bandwidth utilization efficiency.
[0064] Different data sharding strategies are implemented for each caching layer: the first-level cached data is not sharded at all to ensure atomic data transmission; the second-level cached data is logically sharded according to the natural boundaries of business objects, with each shard not exceeding 16,000 bytes; the third-level cached data adopts a fixed-size sharding method, with each shard being 64,000 bytes, to optimize transmission efficiency.
[0065] Meanwhile, the cloud receiving system initiates a two-way verification mechanism that strictly corresponds to the edge-side cache levels. For high-priority data from the first-level cache, the cloud performs triple verification: transport layer verification (TCP checksum + application layer CRC32), data integrity verification (SHA-256 hash calculation and comparison, with the hash value transmitted separately in the packet header), and business logic verification (checking whether the data value is within a reasonable range, such as whether the temperature sensor reading is between -40°C and 125°C). If verification fails, a retransmission is immediately requested through a dedicated control channel (occupying 5% of the total bandwidth and always kept open). For medium-priority data from the second-level cache, two levels of verification are performed: transport layer verification and selective field verification (verifying only key control parameters and status fields, determined by a predefined list of field importance), covering approximately 40%-60% of the total data. For low-priority data from the third-level cache, basic structure verification (confirming that the data format and length meet expectations) and 16-bit cyclic redundancy check are performed, covering approximately 20%-30% of the total data.
[0066] The cloud system builds and maintains a fine-grained data integrity graph, which is based on correlation coefficients calculated at the edge. The graph is constructed using a directed weighted graph structure, where nodes represent data objects, edges represent relationships, and weights represent the strength of those relationships. The graph records the source cache level and priority score of each data object. Last updated timestamp and integrity status.
[0067] When data inconsistency or corruption is detected, the system performs graph analysis: first, it identifies the affected data nodes, and then it searches for their correlation. For all neighboring nodes with a value >0.7, construct an "information redundancy network"; for data fields that cannot be directly recovered, attempt to estimate possible values from related data through Bayesian inference, with the inference confidence threshold set at 85%; for high-priority data with insufficient inference confidence, immediately initiate an accurate retransmission request, including the specific data ID, timestamp, and reason for verification failure; for medium-priority data, only request retransmission of specific fields that failed verification to reduce bandwidth consumption; for low-priority data, divide the number of corrupted fields by the total number of fields to obtain the percentage of corruption. If the percentage of corruption is <5%, mark it as "acceptable loss" and continue processing; if 5% ≤ percentage of corruption < 20%, add it to the low-priority retransmission queue (delayed processing); only request immediate retransmission when the percentage of corruption is ≥20%.
[0068] To maximize transmission efficiency, a precise collaborative data completion mechanism is implemented between the edge and the cloud. When the cloud identifies missing data, it generates a structured "differential request message" containing the following precise fields: priority category identifier of the missing data, detailed timestamp range, data object type identifier (16-byte UUID), bitmap of the specific missing fields, and the current cloud data version number. The differential request message is transmitted through a highly reliable control channel, using redundant coding to ensure 100% delivery.
[0069] Upon receiving a differential request, the edge device first identifies the corresponding cache level based on priority category. Then, it quickly locates the target data object in the index table using a unique identifier and timestamp, accurately extracting missing fields. For data from different cache levels, the system applies different compression strategies: Level 1 cache data uses only lossless compression algorithms to ensure zero data quality loss, reducing data volume by 30% to 60%; Level 2 cache data uses controlled loss compression, employing different precision controls for different fields, reducing data volume by 60% to 80%; Level 3 cache data uses high compression ratio algorithms, accepting up to 15% controllable information loss, with compression ratios reaching 10:1 to 12:1, significantly reducing the amount of data transmitted.
[0070] In extremely weak network environments, where available bandwidth remains below 50Kbps for 10 seconds, the system activates a refined "core element transmission mode": For first-level cached data, predefined core fields are extracted and transmitted (different sets of core fields are defined for different data types; for example, alarm data retains the alarm ID, level, trigger value, and timestamp, while discarding detailed descriptions and historical comparison data). Typically, core fields account for 23-27% of the original data volume. For second-level cached data, filtering... For associated objects with a resolution greater than 0.85, only key status attributes of these objects (approximately 35% of the total fields) are transmitted; third-level cached data transmission is completely suspended until network bandwidth is restored to at least 100Kbps for more than 30 seconds. A precise "transmission debt record" is maintained for the suspended data, recording the ID of each delayed data object, its original priority, delay time, and expected resumption time, ensuring that transmission can be correctly resumed according to priority and timing after network recovery.
[0071] Both the edge and cloud maintain structured transport status logs, accurately recording detailed information for each data block: source cache level, original priority score, etc. The system includes transmission start and finish timestamps, number of transmission attempts, actual bandwidth used, size ratio before and after compression, and verification result code. Every 60 minutes or every 15 minutes when bandwidth is sufficient, the system exchanges checkpoint information on the control channel, which includes a hash digest tree for each type of data (implemented based on Merkle trees, with each layer of cached data building a separate tree).
[0072] If an inconsistency is detected, a binary search algorithm is used to quickly locate the specific inconsistent node in the Merkle tree, accurately determining the cache level and specific object where the inconsistent data resides. A minimum transfer set (containing only the actually inconsistent data objects) is calculated, and a targeted recovery process is initiated only for this portion of data. Bandwidth allocation still follows the aforementioned allocation principle: Level 1 cache accounts for 50%, Level 2 cache for 30%, and Level 3 cache for 20%. During the recovery transfer process, the system continuously monitors the data consistency recovery percentage for each cache level, prioritizing ensuring 100% consistency for Level 1 cache, then at least 95% consistency for Level 2 cache, and finally over 90% consistency for Level 3 cache.
[0073] In summary, an edge IoT gateway data interruption resumption system and method based on embodiments of the present invention have been elucidated. By deploying a multimodal network perception engine, it achieves comprehensive perception and prediction of network latency, packet loss rate, and link quality index, effectively improving the accuracy of network status monitoring. By constructing a data hierarchy matrix and dynamically adjusting data feature weights based on reinforcement learning algorithms, it achieves intelligent data priority allocation, significantly improving the transmission efficiency of critical data. By employing a hierarchical elastic caching mechanism, it dynamically adjusts the caching strategy according to network fluctuations and data characteristics, optimizing the utilization of storage resources. Through an adaptive bandwidth-aware tiered resumption module, it allocates bandwidth and adjusts the resumption rate according to the current network capacity, achieving efficient data completion and transmission reliability assurance. This significantly improves the reliability, real-time performance, and resource utilization efficiency of data transmission.
Claims
1. A method for resuming interrupted data transmission at the edge IoT gateway during network interruption, characterized in that: include: Deploy a multimodal network awareness engine, combine network latency, packet loss rate and link quality index to calculate network health score and achieve predictive identification of network status; Construct a data hierarchy matrix, extract data features, dynamically adjust feature weights based on reinforcement learning algorithms, and generate data priority scores. When a network anomaly warning is detected, a hierarchical elastic caching mechanism is used to intelligently group newly generated data according to priority and relevance and store it in a hierarchical manner. After the network is restored, the adaptive bandwidth-aware tiered resume module allocates bandwidth resources based on the current network capacity and hierarchical cache priority, adjusts the resume rate, and completes data.
2. The edge IoT gateway data breakpoint resumption method with network interruption adaptation according to claim 1, characterized in that, The multimodal network perception engine includes a network parameter acquisition module, a state fusion calculation module, and a prediction and early warning module.
3. The edge IoT gateway data breakpoint resumption method with network interruption adaptation according to claim 2, characterized in that, The prediction and early warning module is based on an ultra-lightweight heartbeat frame mechanism designed with an improved MQTT protocol. It combines round-trip delay jitter mode and loss situation to construct a network fluctuation prediction model and uses a long short-term memory network to output the probability distribution of network fluctuations in the next 800 milliseconds.
4. The edge IoT gateway data breakpoint resumption method with network interruption adaptation according to claim 3, characterized in that, The data grading matrix extracts four features of data: timeliness, criticality, business relevance, and security level. Based on reinforcement learning algorithm, the feature weights are dynamically adjusted to generate data priority scores. Based on the priority scores, the data is divided into emergency alarm data, real-time control data, and non-real-time statistical data.
5. The edge IoT gateway data breakpoint resumption method with network interruption adaptation according to claim 1, characterized in that, The tiered elastic caching mechanism includes three levels of caching: The first-level cache is a high-speed memory buffer that prioritizes storing emergency alarm data. The second-level cache is the main memory storage area, which stores real-time control data and highly relevant non-real-time statistical data. The third-level cache is a persistent storage area that stores low-priority, non-real-time statistical data.
6. The edge IoT gateway data breakpoint resumption method with network interruption adaptation according to claim 5, characterized in that, The allocation coefficient of the tiered elastic caching mechanism Determined by the Network Volatility Severity Index (NTI), the formula is as follows: ,in, This is a network fluctuation severity index.
7. The edge IoT gateway data breakpoint resumption method with network interruption adaptation according to claim 6, characterized in that, The network fluctuation severity index The formula is expressed as: , in, The standard deviation of the round-trip time of a heartbeat frame. This represents the average round-trip time of a heartbeat frame. Heartbeat frame loss rate, The rate of change of round-trip time, This is the normalized value of the network health score. These are the weighting coefficients.
8. The edge IoT gateway data breakpoint resumption method with network interruption adaptation according to claim 7, characterized in that, Based on the aforementioned three-level caching hierarchy, to organize cached data more efficiently, an improved weighted cosine similarity algorithm is used to calculate the correlation coefficient between data. The formula is expressed as: ,in, and Representing data respectively and data In the Values on dimensional features, For the first Weights of dimensional features Indicates the first Information gain of dimensional features This represents the total number of feature dimensions.
9. The edge IoT gateway data breakpoint resumption method with network interruption adaptation according to claim 1, characterized in that, The hierarchical transmission rules of the resume module are as follows: Emergency alarm data in the first-level cache uses a single-packet confirmation mechanism; The real-time control data in the second-level cache uses a dynamically adjusted sliding window mechanism. Non-real-time statistics in the third-level cache are transmitted in batch mode.
10. A network interruption adaptive edge IoT gateway data breakpoint resumption system, characterized in that, include: The network awareness module is used to calculate the network health score and achieve predictive identification of network status by combining network latency, packet loss rate and link quality index. The data classification module is used to extract data features, dynamically adjust feature weights based on reinforcement learning algorithms, and generate data priority scores. The hierarchical caching module is used to intelligently group and store newly generated data in a hierarchical elastic caching mechanism according to priority and relevance when a network anomaly warning is detected. The tiered resume module is used to allocate bandwidth resources based on the current network capacity and hierarchical cache priority, adjust the resume rate, and complete data.