Data transmission system for edge network cloud

Abstract

The invention discloses a data transmission system for edge network cloud, and relates to the field of cloud computing. The data transmission system for the edge network cloud comprises an edge node layer, a network transmission layer and a cloud center layer which work cooperatively through standardized interfaces in sequence, the edge node layer comprises at least one edge node, and each edge node is provided with an edge computing power scheduling unit, a data preprocessing unit and a local cache unit; the network transmission layer is configured with a dynamic routing unit, a QoS scheduling unit and an end-to-end encryption unit; and the cloud center layer is configured with a global resource scheduling unit, a data analysis unit and a security management unit. According to the data transmission system for the edge network cloud, the tasks are dynamically unloaded to the adjacent nodes or the cloud center, overload of a single node is avoided, the overall processing efficiency of the edge layer is improved, and the reasonability and timeliness of task unloading are ensured through the preset threshold value and the distance priority strategy.

Description

Technical Field

[0001] This invention relates to the field of cloud computing, and in particular to a data transmission system for edge cloud networks. Background Technology

[0002] With the rapid development of the Internet of Things (IoT), the Industrial Internet, and 5G technologies, the massive amounts of data generated by terminal devices require real-time processing and transmission. The high latency and high bandwidth consumption problems caused by centralized data processing in traditional cloud computing models are becoming increasingly prominent. Edge computing, as a supplement to cloud computing, offloads some data processing tasks to network edge nodes, which can effectively reduce transmission latency. However, edge nodes have limited computing power and are distributed widely, making them prone to load imbalance.

[0003] Existing edge-cloud collaborative data transmission systems have the following shortcomings: The rigid scheduling of computing power at edge nodes makes it impossible to dynamically unload tasks based on real-time load, leading to overload or waste of resources on some nodes. Fixed network transmission paths make it difficult to adapt to the dynamic changes in complex network environments, and the lack of differentiated quality of service (QoS) guarantees for different types of data can easily cause delays or blockages in critical data transmission. Insufficient resource coordination between the edge and cloud centers makes it impossible to optimize resource allocation based on overall load, resulting in low global resource utilization. The data transmission security and privacy protection mechanisms are not perfect, and there are risks such as insecure key distribution and lack of identity authentication. Its weak multi-protocol adaptability makes it difficult to be compatible with various heterogeneous networks such as 5G, WiFi 6, and LoRa, which limits the deployment and application of the system in complex scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a data transmission system for edge cloud networks to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a data transmission system for edge cloud networks, comprising an edge node layer, a network transmission layer, and a cloud center layer that work collaboratively through standardized interfaces in sequence. The edge node layer includes at least one edge node. Each edge node is configured with an edge computing power scheduling unit, a data preprocessing unit, and a local cache unit. The edge computing power scheduling unit determines whether to unload the data processing task based on the load status of the edge node. The network transport layer is configured with a dynamic routing unit, a QoS scheduling unit and an end-to-end encryption unit. The dynamic routing unit determines and selects the optimal transmission path based on the real-time network status, and the QoS scheduling unit determines and allocates data transmission priority based on the data type. The cloud center layer is configured with a global resource scheduling unit, a data analysis unit, and a security management unit. The global resource scheduling unit realizes the collaborative allocation of resources between the edge and the cloud based on the overall load status of the edge node layer and the cloud center layer.

[0006] Preferably, the edge computing power scheduling unit collects the CPU utilization and memory utilization of edge nodes in real time. When the CPU utilization is greater than a preset threshold A or the memory utilization is greater than a preset threshold B, it automatically determines the load of adjacent edge nodes: if the load of an adjacent node is less than a preset threshold C, then some data processing tasks are offloaded to that adjacent node; if the load of an adjacent node is greater than or equal to the preset threshold C, then the tasks are offloaded to the cloud center layer; the preset threshold A is 80% by default, the preset threshold B is 85% by default, and the preset threshold C is 60% by default, and when offloading tasks, adjacent edge nodes with a distance of less than 1km are selected first.

[0007] Preferably, the dynamic routing unit collects bandwidth, latency, and packet loss rate parameters of the transmission path. When any parameter exceeds the corresponding preset threshold, the path optimization algorithm is triggered to recalculate the optimal path, and the path switching time is ≤100ms. Before path switching, the availability of the backup path needs to be verified. The backup path needs to meet the following conditions: bandwidth ≥ preset threshold D, latency ≤ preset threshold E, and packet loss rate ≤ preset threshold F. The preset thresholds D, E, and F can be customized based on the application scenario.

[0008] Preferably, the QoS scheduling unit determines the data type by the data header identifier and assigns high, medium, and low priorities based on the type (real-time control data is high priority, business interaction data is medium priority, and historical statistics data is low priority). It uses a weighted fair queue algorithm to allocate bandwidth. When the proportion of high-priority data traffic is greater than 80%, the proportion of medium-priority data bandwidth is temporarily increased to 20%-30% to avoid long-term blocking of low-priority data.

[0009] Preferably, the global resource scheduling unit adopts an adaptive resource scheduling algorithm to collect the average load of the edge node layer and the load of the cloud center layer at a preset period: if the average edge load > preset threshold G and the cloud load < preset threshold H for N consecutive periods, then 10%-20% of the computing power of the cloud center is scheduled to the edge layer; if the average edge load < preset threshold I and the cloud load > preset threshold J for N consecutive periods, then 30%-40% of the idle computing power of the edge layer is scheduled to the cloud center; the preset period is 5 minutes by default, N is 3 by default, and the preset thresholds G, H, I, and J are 75%, 50%, 40%, and 80% by default.

[0010] Preferably, the end-to-end encryption unit uses a symmetric encryption algorithm to encrypt the transmitted data and combines it with an asymmetric encryption algorithm to distribute the encryption key; specifically, it distributes the AES symmetric encryption key through the RSA asymmetric algorithm, and before distribution, it first judges the validity of the edge node's identity certificate (if the certificate has not expired and the signature matches the preset signature, the key is distributed; otherwise, the data transmission is rejected).

[0011] Preferably, the local caching unit determines the network status by detecting whether M consecutive data transmissions fail (if M consecutive failures occur, the network is considered interrupted), and prioritizes transmitting cached data after the network recovers; before transmission, it also determines the timeliness of the cached data, deletes historical data that exceeds the preset time, and retains real-time data; M is 3 by default, and the preset time is 24 hours by default.

[0012] Preferably, the data preprocessing unit performs filtering (removing redundant and invalid data), compression (using a lightweight data compression algorithm), and desensitization (masking sensitive fields such as device number and geographical location) on the collected raw data; the lightweight data compression algorithm is specifically the LZ77 algorithm.

[0013] Preferably, the security management unit stores the identity certificates of all edge nodes and completes two-way identity authentication between the edge nodes and the cloud center layer through certificate verification; at the same time, it records a full data transmission log, the log content of which includes at least the data transmission time, data type, sending / receiving node identifier, and transmission path information, for data transmission traceability and anomaly investigation.

[0014] Preferably, the network transmission layer is also configured with a multi-protocol adaptation unit that supports at least three transmission protocols: 5G, WiFi 6, fiber optic, and LoRa. This unit automatically selects the optimal transmission protocol based on the physical distance between the edge node and the cloud center, real-time bandwidth requirements, and transmission costs. When the bandwidth of the current protocol fluctuates by more than 20%, it triggers a protocol reselection.

[0015] The technical effects and advantages of this invention are as follows: The edge computing power scheduling unit monitors node load in real time and dynamically offloads tasks to adjacent nodes or the cloud center to avoid overloading a single node and improve the overall processing efficiency of the edge layer. Preset thresholds and distance-first strategies ensure the rationality and timeliness of task offloading. The dynamic routing unit adaptively selects the optimal path based on real-time network conditions (bandwidth, latency, packet loss rate), with path switching time ≤100ms and verification of backup path availability to ensure transmission continuity. The QoS scheduling unit uses a three-level priority division and a weighted fair queue algorithm to ensure the transmission quality of critical data such as real-time control data and avoid long-term blocking of low-priority data. The global resource scheduling unit uses an adaptive algorithm to periodically adjust computing power allocation based on the overall load status of the edge and cloud. When the edge load is too high, cloud resources are scheduled to supplement it; when the cloud load is too high, idle edge computing power is utilized to improve global resource utilization.

[0016] The end-to-end encryption unit combines RSA and AES algorithms to achieve secure key distribution and data encryption. The security management unit implements two-way authentication through identity certificate verification and records a full transmission log, providing support for data traceability and anomaly investigation, ensuring secure data transmission throughout the entire process. The multi-protocol adaptation unit supports multiple transmission protocols such as 5G, WiFi 6, fiber optic, and LoRa. It can automatically select the optimal protocol based on distance, bandwidth requirements, and cost, and triggers a reselection when bandwidth fluctuations exceed 20%, ensuring stable system operation in heterogeneous network environments. The data preprocessing unit reduces redundant data transmission and protects sensitive information through filtering, compression (LZ77 algorithm), and de-identification. The local caching unit temporarily stores data during network interruptions and prioritizes the transmission of valid data upon recovery, avoiding data loss and duplicate transmissions. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall architecture of the present invention; Figure 2 This is a schematic diagram of the edge node layer architecture of the present invention; Figure 3 This is a schematic diagram of the network transport layer architecture of the present invention; Figure 4 This is a schematic diagram of the cloud center layer architecture of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention provides, for example Figures 1-4The data transmission system shown includes an edge network cloud, an edge node layer, a network transport layer, and a cloud center layer that work together sequentially through standardized interfaces. The edge node layer includes at least one edge node. Each edge node is configured with an edge computing power scheduling unit, a data preprocessing unit, and a local cache unit. The edge computing power scheduling unit determines whether to unload the data processing task based on the load status of the edge node. The network transport layer is configured with a dynamic routing unit, a QoS scheduling unit and an end-to-end encryption unit. The dynamic routing unit determines and selects the optimal transmission path based on the real-time network status, and the QoS scheduling unit determines and allocates data transmission priority based on the data type. The cloud center layer is configured with a global resource scheduling unit, a data analysis unit, and a security management unit. The global resource scheduling unit realizes the collaborative allocation of resources between the edge and the cloud based on the overall load status of the edge node layer and the cloud center layer.

[0020] Furthermore, the edge computing power scheduling unit collects the CPU utilization and memory utilization of edge nodes in real time. When the CPU utilization is greater than the preset threshold A or the memory utilization is greater than the preset threshold B, it automatically determines the load of adjacent edge nodes: if the load of an adjacent node is less than the preset threshold C, then some data processing tasks are offloaded to that adjacent node; if the load of an adjacent node is greater than or equal to the preset threshold C, then the tasks are offloaded to the cloud center layer. The preset threshold A is 80% by default, the preset threshold B is 85% by default, and the preset threshold C is 60% by default. When offloading tasks, adjacent edge nodes with a distance of less than 1km are selected first.

[0021] Furthermore, the dynamic routing unit collects bandwidth, latency, and packet loss rate parameters of the transmission path. When any parameter exceeds the corresponding preset threshold, the path optimization algorithm is triggered to recalculate the optimal path, and the path switching time is ≤100ms. Before switching paths, the availability of backup paths must be verified. Backup paths must meet the following requirements: bandwidth ≥ preset threshold D, latency ≤ preset threshold E, and packet loss rate ≤ preset threshold F. The preset thresholds D, E, and F can be customized based on the application scenario.

[0022] Furthermore, the QoS scheduling unit determines the data type by the data header identifier and assigns high, medium, and low priorities based on the type (real-time control data is high priority, business interaction data is medium priority, and historical statistics data is low priority). It uses a weighted fair queue algorithm to allocate bandwidth. When the proportion of high-priority data traffic is greater than 80%, the proportion of medium-priority data bandwidth is temporarily increased to 20%-30% to avoid long-term blocking of low-priority data.

[0023] Furthermore, the global resource scheduling unit adopts an adaptive resource scheduling algorithm to collect the average load of the edge node layer and the load of the cloud center layer at a preset period: if the average edge load > preset threshold G and the cloud load < preset threshold H for N consecutive periods, then 10%-20% of the computing power of the cloud center is scheduled to the edge layer; if the average edge load < preset threshold I and the cloud load > preset threshold J for N consecutive periods, then 30%-40% of the idle computing power of the edge layer is scheduled to the cloud center; the preset period is 5 minutes by default, N is 3 by default, and the preset thresholds G, H, I, and J are 75%, 50%, 40%, and 80% by default.

[0024] Furthermore, the end-to-end encryption unit uses a symmetric encryption algorithm to encrypt the transmitted data and combines it with an asymmetric encryption algorithm to distribute the encryption key; specifically, it distributes the AES symmetric encryption key through the RSA asymmetric algorithm, and before distribution, it first judges the validity of the edge node's identity certificate (if the certificate has not expired and the signature matches the preset signature, the key is distributed; otherwise, the data transmission is rejected).

[0025] Furthermore, the local caching unit determines the network status by detecting whether M consecutive data transmissions fail (if M consecutive failures occur, the network is considered interrupted). After the network recovers, cached data is transmitted first. Before transmission, the timeliness of the cached data is also determined, historical data exceeding the preset duration is deleted, and real-time data is retained. M is 3 by default, and the preset duration is 24 hours by default.

[0026] Furthermore, the data preprocessing unit performs filtering (removing redundant and invalid data), compression (using a lightweight data compression algorithm), and desensitization (masking sensitive fields such as device number and geographical location) on the collected raw data; the lightweight data compression algorithm is specifically the LZ77 algorithm.

[0027] Furthermore, the security management unit stores the identity certificates of all edge nodes and completes two-way identity authentication between the edge nodes and the cloud center layer through certificate verification; at the same time, it records a full log of data transmission, the log content of which includes at least the data transmission time, data type, sending / receiving node identifier, and transmission path information, for data transmission traceability and anomaly investigation.

[0028] Furthermore, the network transmission layer is also configured with a multi-protocol adaptation unit that supports at least three transmission protocols: 5G, WiFi 6, fiber optic, and LoRa. This unit automatically selects the optimal transmission protocol based on the physical distance between the edge node and the cloud center, real-time bandwidth requirements, and transmission costs. When the bandwidth of the current protocol fluctuates by more than 20%, it triggers a protocol reselection.

[0029] The edge computing power scheduling unit monitors node load in real time and dynamically offloads tasks to adjacent nodes or the cloud center to avoid overloading a single node and improve the overall processing efficiency of the edge layer. Preset thresholds and distance-first strategies ensure the rationality and timeliness of task offloading. The dynamic routing unit adaptively selects the optimal path based on real-time network conditions (bandwidth, latency, packet loss rate), with path switching time ≤100ms and verification of backup path availability to ensure transmission continuity. The QoS scheduling unit uses a three-level priority division and a weighted fair queue algorithm to ensure the transmission quality of critical data such as real-time control data and avoid long-term blocking of low-priority data. The global resource scheduling unit uses an adaptive algorithm to periodically adjust computing power allocation based on the overall load status of the edge and cloud. When the edge load is too high, cloud resources are scheduled to supplement it; when the cloud load is too high, idle edge computing power is utilized to improve global resource utilization.

[0030] The end-to-end encryption unit combines RSA and AES algorithms to achieve secure key distribution and data encryption. The security management unit implements two-way authentication through identity certificate verification and records a full transmission log, providing support for data traceability and anomaly investigation, ensuring secure data transmission throughout the entire process. The multi-protocol adaptation unit supports multiple transmission protocols such as 5G, WiFi 6, fiber optic, and LoRa. It can automatically select the optimal protocol based on distance, bandwidth requirements, and cost, and triggers a reselection when bandwidth fluctuations exceed 20%, ensuring stable system operation in heterogeneous network environments. The data preprocessing unit reduces redundant data transmission and protects sensitive information through filtering, compression (LZ77 algorithm), and de-identification. The local caching unit temporarily stores data during network interruptions and prioritizes the transmission of valid data upon recovery, avoiding data loss and duplicate transmissions.

[0031] Working principle: 1. Data processing and task offloading at the edge node layer: After edge nodes collect raw data, the data preprocessing unit performs filtering (redundancy removal), compression (LZ77 algorithm), and desensitization (masking sensitive fields) to reduce the amount of data transmitted and protect privacy. Simultaneously, the edge computing power scheduling unit monitors node CPU utilization (default threshold A = 80%) and memory utilization (default threshold B = 85%) in real time. If either indicator exceeds the limit: Prioritize determining the load of adjacent nodes within a distance of less than 1km. If the load of an adjacent node is less than the threshold C (default 60%), then offload some tasks to that node. If the load of an adjacent node is greater than or equal to the threshold C, the task will be offloaded to the cloud center layer.

[0032] The local cache unit monitors the data transmission status. If it fails three times in a row, it determines that the network is interrupted and temporarily stores the data. After the network is restored, invalid data older than 24 hours is deleted, and real-time cached data is transmitted first.

[0033] 2. Data transmission control at the network transport layer: The preprocessed data enters the network transmission layer. The multi-protocol adaptation unit selects the optimal solution from protocols such as 5G, WiFi 6, fiber optic, and LoRa based on the physical distance between the edge node and the cloud center, bandwidth requirements, and transmission costs. When the bandwidth of the current protocol fluctuates by more than 20%, a new solution is selected.

[0034] The dynamic routing unit collects path bandwidth, latency, and packet loss rate in real time. When any parameter exceeds the limit, the path optimization algorithm is triggered, and the system switches to an alternative path that meets the preset threshold (D, E, and F can be customized) within ≤100ms to ensure transmission continuity. The QoS scheduling unit distinguishes data types (real-time control data, business interaction data, and historical statistics data) by using data header identifiers, assigns high, medium, and low priorities, and uses a weighted fair queue algorithm to allocate bandwidth. When the proportion of high-priority traffic is greater than 80%, the bandwidth of medium-priority traffic is temporarily increased to 20%-30% to avoid blocking of low-priority data. The end-to-end encryption unit first verifies the edge node's identity certificate (not expired and with a matching signature) using the RSA algorithm. After successful verification, it distributes the AES symmetric encryption key to encrypt the transmitted data and ensure data security.

[0035] 3. Global management and collaboration at the cloud center layer: After encrypted data is transmitted to the cloud center layer, the global resource scheduling unit monitors the average load at the edge and the load at the cloud center for three consecutive cycles with a 5-minute interval. If the average edge load is greater than the threshold G (75%) and the cloud load is less than the threshold H (50%), allocate 10%-20% of the computing power from the cloud center to the edge layer; If the average load at the edge is less than threshold I (40%) and the cloud load is greater than threshold J (80%), 30%-40% of the idle computing power at the edge layer is scheduled to the cloud center to achieve global resource coordination.

[0036] The data analysis unit performs in-depth analysis of the received data, and the security management unit stores all edge node certificates and records transmission logs (including time, type, node identifier, path, and other information) for identity authentication, data traceability, and anomaly investigation.

[0037] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data transmission system for edge cloud networks, characterized in that, This includes the edge node layer, network transport layer, and cloud center layer, which work together sequentially through standardized interfaces: The edge node layer includes at least one edge node. Each edge node is configured with an edge computing power scheduling unit, a data preprocessing unit, and a local cache unit. The edge computing power scheduling unit determines whether to unload the data processing task based on the load status of the edge node. The network transport layer is configured with a dynamic routing unit, a QoS scheduling unit and an end-to-end encryption unit. The dynamic routing unit determines and selects the optimal transmission path based on the real-time network status, and the QoS scheduling unit determines and allocates data transmission priority based on the data type. The cloud center layer is configured with a global resource scheduling unit, a data analysis unit, and a security management unit. The global resource scheduling unit realizes the collaborative allocation of resources between the edge and the cloud based on the overall load status of the edge node layer and the cloud center layer.

2. The data transmission system for edge cloud as described in claim 1, characterized in that, The edge computing power scheduling unit collects the CPU utilization and memory utilization of edge nodes in real time. When the CPU utilization is greater than the preset threshold A or the memory utilization is greater than the preset threshold B, it automatically determines the load of adjacent edge nodes: if the load of the adjacent node is less than the preset threshold C, some data processing tasks are offloaded to the adjacent node; if the load of the adjacent node is greater than or equal to the preset threshold C, the tasks are offloaded to the cloud center layer. The preset threshold A is 80% by default, the preset threshold B is 85% by default, and the preset threshold C is 60% by default. When offloading tasks, adjacent edge nodes with a distance of less than 1km are selected first.

3. The data transmission system for edge cloud as described in claim 1, characterized in that, The dynamic routing unit collects bandwidth, latency, and packet loss rate parameters of the transmission path. When any parameter exceeds the corresponding preset threshold, the path optimization algorithm is triggered to recalculate the optimal path, and the path switching time is ≤100ms. Before path switching, the availability of the backup path needs to be verified. The backup path must meet the following requirements: bandwidth ≥ preset threshold D, latency ≤ preset threshold E, and packet loss rate ≤ preset threshold F. The preset thresholds D, E, and F can be customized based on the application scenario.

4. A data transmission system for edge cloud as described in claim 1, characterized in that, The QoS scheduling unit determines the data type by the data header identifier, allocates high, medium and low priorities based on the type, and uses a weighted fair queue algorithm to allocate bandwidth. When the proportion of high priority data traffic is greater than 80%, the proportion of medium priority data bandwidth is temporarily increased to 20%-30% to avoid long-term blocking of low priority data.

5. A data transmission system for edge cloud networks according to claim 1, characterized in that, The global resource scheduling unit adopts an adaptive resource scheduling algorithm, which collects the average load of the edge node layer and the load of the cloud center layer at a preset period: if the average edge load > preset threshold G and the cloud load < preset threshold H for N consecutive periods, then 10%-20% of the computing power of the cloud center is scheduled to the edge layer; if the average edge load < preset threshold I and the cloud load > preset threshold J for N consecutive periods, then 30%-40% of the idle computing power of the edge layer is scheduled to the cloud center; the preset period is 5 minutes by default, N is 3 by default, and the preset thresholds G, H, I, and J are 75%, 50%, 40%, and 80% by default.

6. A data transmission system for edge cloud as described in claim 1, characterized in that, The end-to-end encryption unit uses a symmetric encryption algorithm to encrypt the transmitted data and combines it with an asymmetric encryption algorithm to distribute the encryption key; specifically, it distributes the AES symmetric encryption key through the RSA asymmetric algorithm, and first determines the validity of the edge node's identity certificate before distribution.

7. A data transmission system for edge cloud networks according to claim 1, characterized in that, The local caching unit determines the network status by detecting whether M consecutive data transmissions have failed. After the network recovers, it prioritizes transmitting cached data. Before transmission, it also determines the timeliness of the cached data, deletes historical data that exceeds the preset duration, and retains real-time data. M is 3 by default, and the preset duration is 24 hours by default.

8. A data transmission system for edge cloud as described in claim 1, characterized in that, The data preprocessing unit performs filtering, compression, and desensitization processing on the collected raw data.

9. A data transmission system for edge cloud as described in claim 1, characterized in that, The security management unit stores the identity certificates of all edge nodes and completes two-way identity authentication between edge nodes and the cloud center layer through certificate verification. At the same time, it records a full log of data transmission, which includes at least the data transmission time, data type, sending / receiving node identifier, and transmission path information for data transmission traceability and anomaly investigation.

10. A data transmission system for edge cloud as described in claim 1, characterized in that, The network transmission layer is also configured with a multi-protocol adaptation unit that supports at least three transmission protocols: 5G, WiFi 6, fiber optic, and LoRa. This unit automatically selects the optimal transmission protocol based on the physical distance between the edge node and the cloud center, real-time bandwidth requirements, and transmission costs. When the bandwidth of the current protocol fluctuates by more than 20%, it triggers a protocol reselection.

Images