An industrial internet of things edge cloud collaborative dynamic scheduling and twin data consistency synchronization method based on heterogeneous computing power perception

Abstract

This invention discloses a method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power awareness, relating to the fields of industrial internet and distributed computing technology. The method includes the following steps: collecting parameters such as central processing resource occupancy, available computing power frequency, network round-trip latency, and link throughput of edge nodes, and constructing a multi-dimensional resource parameter set reflecting the node's operating status by combining historical data prediction results. This invention achieves dynamic task scheduling by integrating computing power and network status, ensuring stable execution of critical tasks and optimizing resource utilization in complex environments. Simultaneously, it achieves incremental data synchronization through a hierarchical hash structure, reducing transmission overhead and improving synchronization efficiency in weak network and network outage recovery scenarios, thereby enhancing the overall system stability and data consistency assurance capabilities.

Description

Technical Field

[0001] This invention relates to the field of industrial internet and distributed computing technology, specifically to an industrial IoT edge-cloud collaborative dynamic scheduling and twin data consistency synchronization method based on heterogeneous computing power perception. Background Technology

[0002] Industrial Internet of Things (IIoT) systems are gradually evolving from centralized cloud computing architectures to distributed architectures that combine edge computing and cloud computing. In this architecture, a large number of industrial devices collect, preprocess, and control data in real time through edge nodes, and then perform global analysis and management through a cloud platform. Current mainstream implementations typically deploy and schedule applications based on containerization technology and cluster scheduling frameworks (such as Kubernetes), and combine them with device access middleware to achieve unified management of industrial devices. However, due to the heterogeneous computing power of devices in industrial settings (such as mixed deployment of ARM and x86) and the complex and variable network environment (such as fluctuations in 4G / 5G link bandwidth and significant latency jitter), traditional scheduling mechanisms based on static resource configuration and simple data synchronization methods are insufficient to meet the high requirements for real-time performance and reliability in industrial scenarios.

[0003] Existing technologies still have significant shortcomings in edge-cloud collaborative scheduling and data synchronization: On the one hand, existing scheduling mechanisms mainly rely on static resource thresholds such as CPU and memory, lacking the ability to dynamically perceive changes in real-time computing power and network status. This results in critical real-time tasks being scheduled to be executed in the cloud even when there is network congestion or increased edge load, leading to response delays or even task failures. On the other hand, existing edge-cloud data synchronization mechanisms mostly adopt full data transmission or simple log synchronization methods. In weak network or network outage environments, this not only wastes bandwidth resources but also easily leads to data inconsistencies or even split-brain problems after the network is restored. The lack of efficient difference location and conflict resolution mechanisms makes it difficult to guarantee the accuracy and consistency of industrial digital twin data.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception, so as to solve the problems in the background technology mentioned above.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception, comprising the following steps: Collect parameters such as central processing resource usage, available computing power frequency, network round-trip latency, and link throughput of edge nodes, and construct a multi-dimensional resource parameter set reflecting the node's operating status by combining historical data prediction results; The execution cost function of the task on different nodes is calculated based on the multi-dimensional resource parameter set. The tasks are classified according to the task latency constraints. The scheduling position of the task between edge nodes or cloud nodes is determined by combining the execution cost function and the task classification result. When the scheduling location changes, the running context of the corresponding task is frozen and a state snapshot is generated. The state snapshot is then migrated to the node corresponding to the scheduling location and restored, while maintaining the continuity of network connection status during task execution. For the twin data generated during task execution, it is divided into layers according to device identifier, business attributes and data fields, and the corresponding hash value is calculated based on the data of each layer to construct a multi-layer hash structure; The root hash value of the multi-level hash structure is compared. When the root hash values ​​are inconsistent, the difference data is located layer by layer and only the difference data is transmitted. At the same time, the corresponding conflict determination rules are executed according to the data attribute type to complete the data consistency synchronization.

[0007] Preferably, to address the problem of characterizing dynamic fluctuation features in the representation of node operating status, a multi-dimensional parameter fusion processing logic is introduced, with the following steps: Data on central processing resource usage, memory usage, and network round-trip latency are collected according to a preset sampling period and then constructed as continuous time series data in chronological order. The historical data of link throughput is processed using a sliding window, and short-term forecast values ​​are calculated using the exponential moving average algorithm. The system performs dispersion analysis on the time series of central processing resource usage to calculate load jitter index, and simultaneously calculates the variance of network packet loss rate changes to generate link fluctuation index. Time series data, throughput forecasts, and various volatility indicators are uniformly normalized and combined to form a multi-dimensional resource parameter set.

[0008] Preferably, a multi-factor coupled calculation mechanism is constructed around the matching relationship between task execution cost assessment and scheduling decision, with the following steps: Read the data scale information involved in the task processing process and the processing cycle data in the historical execution records, and construct a task feature set; The time required for data transmission is calculated based on the link throughput capacity, and the task execution time is calculated based on the frequency of available computing power of the node. Tasks are divided into different levels according to task latency constraints, and corresponding weight coefficients are assigned to each level. The execution cost function is obtained by weighted summation of the transmission time term and the execution time term, and the scheduling position is output accordingly.

[0009] Preferably, to address the stability control requirements related to task migration triggering timing, a persistence determination constraint mechanism is introduced, with the following steps: The usage of central processing resources is continuously sampled and the change curves within multiple sampling periods are recorded. Statistical analysis of network egress queue depth and round-trip latency is performed, and a link status change sequence is generated. It determines whether the central processing resource usage continuously exceeds a preset threshold, and detects the duration of continuous abnormal link status. When both abnormal resource usage and abnormal link status meet the persistence condition, a migration trigger signal is generated and a scheduling control command is output.

[0010] Preferably, the curve of central processing resource occupancy change is smoothed by sliding window to form a stable trend sequence. At the same time, the fluctuation amplitude of the link state change sequence is filtered to obtain the effective state interval. The stable trend sequence and the effective state interval are combined for joint matching and judgment to generate a migration trigger signal and output scheduling control instructions.

[0011] Preferably, to address the requirement of maintaining the integrity of the running state during task migration, a context freezing and snapshot generation processing method is designed, with the following steps: The user-mode checkpoint mechanism is invoked to obtain the process tree information and execution context of the running task; Serialize the memory page data, file descriptors, and thread states involved in the process execution to generate state data; Capture and record network connection information, including communication endpoints, connection status, and buffer data; The execution context data, memory data, and network connection status are integrated to generate migration snapshot data.

[0012] Preferably, in combination with the business continuity assurance requirements after cross-node migration, an operation recovery and communication maintenance mechanism is constructed, with the following steps: The migration snapshot data is transmitted to the target node over the network and data integrity is verified. Perform deserialization on the snapshot data and restore the process execution context and memory mapping structure; Identify existing communication connections and establish corresponding forwarding rules and connection mapping tables; The network forwarding table is updated and communication path redirection is completed through kernel-mode data processing technology.

[0013] Preferably, based on the requirements for structured modeling of twin data, a hierarchical hash construction and summary generation logic is designed, with the following steps: Group the twin data according to the device identifier and construct device-level data nodes; The data in each device node is divided into a telemetry data set and a configuration data set according to business attributes; Extract the field identifier, data content, and time information from each data field to construct a leaf node data unit; Perform hash calculations on the leaf nodes and aggregate them level by level according to the hierarchical relationship to generate a multi-level hash structure.

[0014] Preferably, the leaf node hash value is concatenated with field identifier, field data content and corresponding time information to form input data, and a unified hash algorithm is used to generate a digest value. The internal nodes generate the upper-level node digest by combining the child node hash values ​​in a fixed order, forming a bottom-up consistency verification structure.

[0015] Preferably, to address the issues of difference identification and conflict handling during data synchronization, an incremental synchronization and consistency control strategy is constructed, with the following steps: Exchange the root node digests of the multi-level hash structure and perform a consistency check; When there are differences in the root node digest, the hash values ​​of each node are compared level by level to locate the difference path; Extract the corresponding leaf node data based on the difference path and classify the data type according to data attributes; Apply the corresponding conflict determination rules to different types of data and complete the transmission of the difference data.

[0016] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This invention constructs a dynamic scheduling mechanism based on joint awareness of computing power and network status, enabling adaptive adjustment of task execution location according to the real-time operating status of nodes in complex industrial environments. When edge resources are limited or network status fluctuates, it can prioritize the stable operation of latency-sensitive tasks while rationally guiding computing load to the cloud, thereby improving overall resource utilization efficiency and reducing task execution latency. This allows the system to maintain good operational continuity and responsiveness even under heterogeneous multi-device conditions.

[0017] This invention, by introducing a hierarchical hash structure and a difference location mechanism, transforms data synchronization from full transmission to incremental transmission, enabling precise synchronization of only changed data in weak network and network outage recovery scenarios. It quickly determines consistency through hash digests and combines this with conflict handling rules to complete data merging, effectively reducing network transmission pressure and improving synchronization efficiency, making data consistency maintenance in large-scale data scenarios more efficient and reliable. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a system architecture diagram of the present invention.

[0020] Figure 2 This is a flowchart of the task dynamic unloading process based on the cost function of this invention.

[0021] Figure 3 This is a schematic diagram illustrating the device shadow incremental synchronization principle based on Merkle trees in this invention.

[0022] Figure 4 This is a timing diagram for data integrity verification in the network outage recovery scenario of this invention. Detailed Implementation

[0023] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0024] This invention provides, for example Figures 1-4 The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception is shown below, with the following steps: In the industrial IoT edge-cloud collaborative system, edge nodes play a key role in connecting field devices with upper-level platforms. Their core capabilities are not only reflected in data acquisition, but also in the refined perception of the operating environment and the pre-processing of data consistency mechanisms.

[0025] Specifically, a lightweight collaborative unit is deployed at the edge, residing on industrial gateways or edge computing devices and tightly coupled with the underlying operating system and container runtime environment. It uses a multi-dimensional probe mechanism to frequently collect local operational status data. The collected data includes not only traditional central processing resource utilization and memory usage, but also more dynamic indicators, such as the degree of computing power fluctuation (i.e., computing power entropy) calculated through a sliding time window, reflecting the stability and predictability of the current node load. Simultaneously, it continuously monitors network link round-trip latency, jitter, packet loss rate, and queue backlog, and uses an exponential moving average algorithm to predict the available throughput of links in a short period, thus forming a joint characterization of the node's computing and network capabilities. This multi-dimensional indicator system can more realistically reflect the operating status of edge nodes in complex industrial environments, providing reliable input for subsequent scheduling decisions.

[0026] Building upon resource awareness, this collaborative unit also undertakes the responsibilities of data change detection and local data modeling. When access devices undergo status updates, configuration changes, or periodic data collection and writing, the edge side does not directly report all data. Instead, it first constructs a structured data organization locally. Data is divided according to a hierarchical logic of device dimension + business attribute + field granularity, breaking down the raw data into multiple data units with clear semantic boundaries. For example, at the device level, different physical devices are distinguished; at the business level, telemetry data and configuration data are distinguished; and at the field level, it is refined to specific collection points or parameter items. This hierarchical structure not only facilitates management but also provides a foundation for subsequent differential calculations.

[0027] After the data structure is constructed, a hashing mechanism is introduced to generate summaries for each layer of nodes. Each smallest data unit is calculated using fixed rules to obtain its corresponding hash value, while the hash value of a higher-level node is calculated by combining the hashes of its child nodes in a predetermined order, thus aggregating layer by layer to form a complete tree-like summary structure. This structure possesses a natural characteristic of local change location; that is, when a field changes, only the path from that field node to the root node is affected, without needing to recalculate the summary result of the entire dataset, significantly reducing computational overhead. Simultaneously, version identifiers are used to mark each layer of data, enabling precise tracking of data changes within different time windows.

[0028] When a data change is detected, the edge does not immediately perform a full synchronization. Instead, it compares the currently generated root digest with historical records to determine if the overall data has changed. If the root digest remains consistent, no transmission operation is required. If a difference exists, the changed path range is further marked, and the identification and version information of the relevant nodes are recorded. At this point, the reported content is no longer the original data, but a summary data containing the difference location information and necessary change descriptions, thereby significantly reducing network transmission pressure.

[0029] The advantages of this mechanism are particularly evident in scenarios with weak networks or limited bandwidth. Traditional methods often require the repeated transmission of large amounts of unchanged data, while by using local hash comparisons and difference markers, only the smallest block of data that has changed needs to be transmitted to complete the synchronization preparation. In addition, in the event of a network outage, the edge can also cache changed data in fragments according to time windows, with each fragment independently maintaining its summary information. This allows data consistency to be gradually restored based on the fragment granularity after network recovery, avoiding sudden traffic surges.

[0030] Furthermore, the edge layer also includes necessary contextual information during the reporting process, such as device identifier, data type identifier, time window number, and version sequence information, to support the upper layer in accurate verification and conflict determination. This design not only improves the efficiency of data synchronization but also provides a basis for subsequent conflict handling, enabling data updates from different sources to be coordinated under unified rules.

[0031] In summary, this mechanism transforms traditional passive data reporting into proactive difference detection and fine-grained synchronization by introducing high-frequency resource awareness, multi-layer data modeling, and hash-based difference detection methods at the edge. This significantly reduces bandwidth consumption while ensuring real-time performance and provides reliable support for data consistency in complex network environments. This design is closely integrated with subsequent scheduling decisions and cloud verification processes, forming a crucial foundational component of the entire edge-cloud collaborative system.

[0032] In the edge-cloud collaborative architecture, the cloud side is responsible for the core of global scheduling and decision-making. The key lies in how to achieve dynamic optimization selection of task execution location based on the multi-dimensional resource status reported by the edge side.

[0033] Specifically, the cloud first receives resource status information continuously reported by each edge node. This information covers key indicators such as central processing resource utilization, actual available computing power frequency, network round-trip latency, and link throughput. Combined with predictions obtained from the edge side using sliding window or exponential moving average methods, the cloud characterizes the short-term performance trends of the nodes. By fusing these multi-dimensional parameters, a comprehensive operational profile of each node at the current moment can be formed. This profile not only reflects the node's immediate load but also its stability and availability in the near future, thus providing a more reliable basis for scheduling decisions.

[0034] Building upon this foundation, a computing potential energy field model is introduced, mapping each computing node in the entire edge-cloud collaborative environment to a topological space with dynamic gradient characteristics. Within this space, different nodes are assigned different potential energy values ​​based on their resource scarcity, network status, and load fluctuations. Typically, nodes with high resource utilization, severe network congestion, or large computing power fluctuations exhibit a high potential energy state, while nodes with sufficient resources and stable networks exhibit a low potential energy state. The scheduling process of tasks within this potential energy field can be analogized to migration behavior within a potential energy gradient; that is, under the premise of satisfying constraints, tasks preferentially move to low potential energy regions, thereby achieving a balanced utilization of overall resources.

[0035] To implement the aforementioned scheduling mechanism, a task-oriented cost function model needs to be constructed. This model comprehensively considers factors such as the data scale, network transmission costs, and computational execution overhead involved in task processing. For example, the amount of data a task needs to process directly affects transmission latency, the available throughput of the link determines data transmission efficiency, and the number of processing cycles required and the frequency of available computing power on the nodes jointly determine the computation completion time. By weighting and combining these factors, the comprehensive cost of the task when executed on different nodes can be obtained. This cost not only reflects execution efficiency but also indirectly reflects the impact of resource consumption on the overall system, thus becoming an important basis for scheduling decisions.

[0036] Meanwhile, to adapt to the varying latency sensitivities of different tasks in industrial scenarios, a task classification mechanism needs to be introduced. Based on the real-time requirements of each task, tasks are divided into multiple levels, such as highly latency-sensitive control tasks, relatively flexible stream processing tasks, and latency-insensitive but computationally intensive batch processing tasks. The weighting of different task levels in the cost function differs; high real-time tasks emphasize minimizing computation and network latency, while low real-time tasks focus more on overall resource utilization efficiency. This differentiated strategy can prevent critical tasks from being interfered with by low-priority tasks, while simultaneously improving the overall system throughput.

[0037] After cost assessment and task classification, the cloud determines the optimal execution location for tasks based on the cost distribution across nodes. When edge nodes are under high load or network conditions deteriorate, some migrateable tasks will be redirected to the cloud or other nodes with better resources for execution; conversely, when network conditions are good and edge computing power is sufficient, tasks can be preferentially retained for local execution to reduce unnecessary transmission overhead. The entire process is not a one-time decision but is dynamically adjusted as resource status continues to change, thus forming a continuously evolving scheduling mechanism.

[0038] Furthermore, to avoid frequent migrations due to momentary fluctuations, scheduling behavior needs to be constrained by persistent criteria. For example, migration decisions should only be triggered when central processing resource usage consistently exceeds a set threshold across multiple consecutive sampling periods, or when network latency and queue depth remain abnormal. This mechanism effectively filters out interference from short-term jitter and improves scheduling stability.

[0039] In summary, by continuously sensing the resource status of edge nodes, introducing a potential energy field model for unified modeling, and combining a multi-factor cost function with a task hierarchical strategy for comprehensive decision-making, dynamic task scheduling for complex industrial environments can be achieved. This method effectively improves overall resource utilization efficiency while ensuring the real-time performance of critical tasks, and can adapt to real-world operating scenarios with constantly changing heterogeneous computing power and network conditions.

[0040] In industrial IoT edge-cloud collaborative scenarios, traditional scheduling methods relying on static resource thresholds struggle to achieve efficient and stable task allocation due to significant differences in heterogeneous node resources and dynamic changes in network status. Therefore, a computing power potential field model is introduced to uniformly model the entire cluster, abstracting each computing node as a potential energy node in a dynamically changing state. By constructing a real-time evaluation function, adaptive decisions are made regarding task execution locations, thereby achieving global resource balancing and performance optimization.

[0041] In this mechanism, a task execution cost function is first defined to quantify the execution cost of a task on a specific node, and its expression is as follows: This formula takes into account both data transmission cost and computation execution cost, and the meanings of each parameter are as follows: This represents the overall cost of executing a task at the target node and is the core basis for scheduling decisions; It is the weighting coefficient of the data transmission part, used to adjust the degree of influence of network factors in the total cost; It is the weighting coefficient of the computational execution part, used to adjust the degree of influence of computing power factors in the total cost; This indicates the size of the instantaneous data block that the current task needs to process, in bytes, and directly affects the transmission time. Indicates time The available throughput of the link at any given time is predicted using the Exponential Moving Average (EMA) algorithm over a sliding window of the past 5 seconds, and is expressed in Mbps. This parameter reflects the dynamic changes in network bandwidth. This represents the number of CPU clock cycles required to execute a task. It is usually estimated using historical execution records and is used to measure the computational complexity of a task. Indicates time The actual available computing power frequency of a node at any given time is measured in GHz. This value is calculated in conjunction with hardware thermal management mechanisms (such as thermal throttling) to accurately reflect the node's current available computing power.

[0042] The aforementioned cost function maps the transmission and computation time costs during task execution to a comparable value, enabling optimal selection among different nodes. When network bandwidth is low or latency is high, the first cost increases significantly; when node computing power is limited or hot-dip throttling occurs, the second cost increases accordingly. The scheduling process selects the execution location with the lowest overall cost based on the differences in this function's value across different nodes.

[0043] Regarding the setting of weighting coefficients, and It is not a fixed value, but rather dynamically adjusted based on task type to adapt to the different real-time and resource consumption requirements of various businesses. Specifically, a three-level task classification mechanism is used for mapping: For L1-level hard real-time tasks, such as equipment closed-loop control or emergency shutdown operations, these tasks are extremely sensitive to latency, typically requiring millisecond-level response times. Therefore, the weighting configuration emphasizes determinism in computation and transmission, employing... Strategies (such as) At the same time, it restricts task migration behavior, fixing it to execute on edge nodes, and ensures response stability by prioritizing resource protection.

[0044] For L2-level soft real-time tasks, such as streaming data preprocessing or protocol parsing, these tasks offer a degree of flexibility and can be dynamically migrated while meeting latency constraints. The weighting coefficients are configured in a relatively balanced manner, ensuring that scheduling decisions consider both network and computing power factors. When the execution cost of edge nodes is higher than that of cloud nodes, i.e., when the latency constraint is met... (If the cost of executing a task on an edge node is greater than the cost of executing it in the cloud, then the task is more suitable to be migrated to the cloud.) After a certain period of time, the task is triggered to migrate to the cloud in order to optimize resources.

[0045] To prevent the scheduling system from overreacting to short-term fluctuations, a triggering mechanism based on a persistent waterline is introduced to constrain migration behavior. Task migration is only executed when resource or network conditions remain consistently abnormal, thus ensuring system stability. Specifically, this includes the following two aspects: First, there is the constraint of central processing resources. When the CPU or memory usage of the edge node exceeds 85% for three consecutive sampling periods (each period is 10 seconds), it indicates that the node is under high load for a long time and needs to alleviate the pressure by migrating some tasks.

[0046] Secondly, network state constraints: when the network queue depth at a node's egress point exceeds the safety threshold for 5 seconds, or when the network round-trip latency remains stable at more than 200% of the baseline value, it indicates that there is significant congestion or performance degradation in the current link. Continuing to execute tasks on this node will lead to a decrease in overall efficiency.

[0047] Through the above mechanism, the task scheduling process can be transformed from traditional static resource matching into an adaptive optimization process based on dynamic environment awareness. Under the influence of the computing power potential field model, each node forms a dynamic gradient distribution, in which tasks migrate along the path of minimum cost, achieving the synergistic goal of stable execution of high-priority tasks and elastic scheduling of low-priority tasks, thereby significantly improving the system's operating efficiency and reliability in complex industrial environments.

[0048] In the process of edge-cloud collaborative operation, the effective implementation of scheduling decisions depends on the control link's precise takeover and migration of task operation status.

[0049] In practice, the control link is deployed between the cloud and edge nodes. By parsing and executing container runtime interface commands, it achieves full lifecycle management of task execution instances. Upon receiving a migration command corresponding to the scheduling result, the currently running container instance is first frozen. The freezing process is based on a user-space checkpoint recovery mechanism, which fully captures information such as process states, memory page contents, and file descriptors involved in the container's operation. At the same time, it marks and records the status of network connections that are being established or maintained, thereby forming a migrateable runtime snapshot.

[0050] After capturing the state, the control link serializes the generated snapshot data and transmits it to the target execution node via the network channel. During transmission, incremental synchronization can be used to transmit only changed memory pages or state fragments, reducing bandwidth consumption and shortening migration time. Upon receiving the snapshot data, the target node deserializes and reconstructs it based on the original runtime context, enabling the container instance to be restored to its pre-migration runtime state on the new node, including key elements such as process execution location, memory mapping relationships, and system call context.

[0051] To ensure business continuity during the migration process, the control link further dynamically adjusts the network communication path. By modifying the underlying network forwarding rules, communication traffic originally directed to local containers on edge nodes is redirected to container instances on the new nodes, thereby achieving seamless connection switching. This process can be combined with kernel-level packet processing technology to hijack and forward connection states, avoiding connection interruptions or data loss due to migration, and ensuring that upper-layer industrial equipment or applications maintain consistent communication awareness before and after the migration.

[0052] Furthermore, during the migration process, it is necessary to track and provide feedback on the migration status, including migration progress, recovery results, and any anomalies, so that the scheduling side can perform subsequent optimization or compensation operations based on the execution results. Through the above mechanism, dynamic migration across nodes can be achieved without affecting the continuous operation of tasks, thereby supporting flexible scheduling and resource optimization in heterogeneous computing environments.

[0053] In the edge-cloud collaborative system, maintaining and verifying the consistency of global virtual device status is a core aspect of ensuring the reliability of industrial data.

[0054] Specifically, the cloud continuously maintains a virtual state set covering all connected devices. This state set exists in the form of a digital twin, fully mapping the configuration parameters, operating status, and historical update information of various devices. Whenever data changes occur at the edge and a local hash calculation is completed, the corresponding root hash value and related version information are reported to the cloud. Upon receiving this digest information, the cloud does not directly perform a full data comparison. Instead, it first performs a rapid verification with the currently maintained global root hash value to determine if there are any discrepancies between the two sets of data. This digest-based preliminary judgment can complete the overall consistency check with extremely low overhead.

[0055] When inconsistencies exist in the root hash value, the cloud initiates a further difference localization process. This process unfolds step-by-step according to a predefined data hierarchy, from the device layer to the business attribute layer, and then to the specific field layer. By comparing the hash values ​​of child nodes layer by layer, the range of differences is quickly narrowed down until the smallest data unit that has changed is located. After localization is complete, the cloud generates precise request information for the differing node and sends a data supplementation request to the edge side via the communication link, obtaining only the necessary changed data to avoid redundant transmission.

[0056] During data backhaul and merging, a conflict handling mechanism based on data attributes is introduced to address potential concurrent modifications from multiple sources. For control data representing desired states, the cloud version is retained first to ensure consistency in system scheduling and control strategies. For monitoring data reflecting real-time device operating status, edge-side data collection is prioritized to ensure data authenticity. For general metadata or data updated from multiple sources simultaneously, timestamps and version sequences are used for comprehensive judgment, and merging is completed using a last-update-priority strategy. This categorized processing approach enables reasonable data adjudication under different business semantics, avoiding information loss or logical conflicts caused by simple overwriting.

[0057] Furthermore, throughout the consistency maintenance process, version numbers and transaction identifiers are used to manage the synchronization process, ensuring the traceability and verifiability of data update order. When a connection is re-established after a network interruption, differential synchronization can continue based on existing version information without requiring a full data verification, thus improving recovery efficiency. Through these mechanisms, the cloud not only achieves unified management of the global device status but also provides efficient and reliable data consistency assurance capabilities in complex network environments.

[0058] In the industrial IoT edge-cloud collaborative environment, to address the issues of network outage resume transmission and difficulty in ensuring data consistency, a multi-layer device twin hash tree structure is introduced. This transforms the original method of relying on full data comparison into a difference detection mechanism based on structured summaries, thereby enabling efficient and reliable data synchronization even in environments with weak networks, jitter, or even intermittent disconnections.

[0059] At the data modeling level, a hierarchical structure is adopted, following a dual partitioning logic of spatial and business isolation. The top layer is the root node, representing the global state of all device data within the entire gateway scope; its hash value serves as a consistency fingerprint for the current data set. The second layer is divided according to the physical device dimension, with each device corresponding to an independent node, naturally decoupling data between different devices. The third layer further partitions data within each device according to business attributes, distinguishing between telemetry data and configuration data to accommodate different update frequencies and business semantics. The bottom layer consists of leaf nodes, corresponding to specific data fields, achieving the finest-grained data expression, such as the real-time value of a single sensor or a specific configuration parameter.

[0060] In terms of hash calculation, each leaf node generates a digest value according to a unified rule, and the calculation formula is as follows: The meanings of each parameter are as follows: It is a unique digest identifier for the data. It is the cryptographic hash algorithm used to generate this digest, which is used to implement data change detection and consistency verification. It is a unique identifier for the device or object to which the data belongs, used to distinguish different data sources; These are field names or attribute identifiers used to describe the business meaning of the data; It is the actual value or content corresponding to the field, and it is the core data for hash calculation; This refers to the data generation or update time, used to distinguish the data status of different time versions. This is achieved by concatenating the above four parameters. Calculation can ensure that the data has strong consistency identification capabilities, that is, a change in any field will cause a significant change in the hash result.

[0061] For non-leaf nodes, their hash value is obtained by concatenating the hash values ​​of all their child nodes in lexicographical order, and then performing a unified hashing process. The calculation yields the result. This layer-by-layer aggregation method gives the hash tree excellent structural propagation properties, meaning that any change in lower-level data will propagate upwards to the root node. Furthermore, when a leaf node changes, only the path from that node to the root node needs to be updated, without recalculating the entire tree, thus keeping the computational complexity within a certain range. in, This represents the total number of leaf nodes. This complexity characteristic allows the method to maintain high computational efficiency even in scenarios involving large-scale devices and high-frequency data updates.

[0062] During data synchronization, consistency is first quickly determined using the root node hash value. The edge and cloud only need to exchange root hash values ​​to determine overall data consistency. When the root hashes match, no synchronization is required; when they do not match, a difference localization process begins. This process unfolds layer by layer based on a tree structure, comparing the hash values ​​of child nodes level by level, starting from the root node, gradually narrowing down the range of differences, and ultimately pinpointing the leaf node where the change occurred. This avoids the high overhead of comparing all data item by item.

[0063] In scenarios involving network outage recovery or concurrent updates across multiple devices, the same field may be modified simultaneously by different nodes, necessitating the introduction of a conflict resolution mechanism. Based on data semantics, data is categorized into three types for processing: For control data representing desired states (Desired), the cloud version is prioritized and overlaid on the edge to ensure consistency of control logic; for monitoring data representing actual states (Reported), edge-side data is prioritized to ensure data authenticity and real-time performance; for public metadata or data without explicit priority, VectorClock and NTP timestamps are introduced. By recording the update order of different nodes and combining it with time information, merging is performed according to the strategy of the last writer winning, thus achieving reasonable data consistency processing.

[0064] After completing difference identification and conflict resolution, the data transmission phase begins. This phase no longer performs full synchronization; instead, it only transmits data from leaf nodes that have changed—that is, only the necessary data blocks and their corresponding hashes and version information are sent. This incremental transmission method significantly reduces network bandwidth consumption, making it particularly suitable for industrial environments with limited link bandwidth or significant network fluctuations. Simultaneously, through version identification and time window mechanisms, the data synchronization process is ensured to be sequential and recoverable. When the connection is restored after a network interruption, synchronization can continue based on the existing state without retransmitting historical data.

[0065] Overall, this mechanism transforms the data consistency problem into a quantifiable, localizable, and optimizable process through a combination of hierarchical modeling, hash digests, difference location, conflict rules, and incremental transmission. While ensuring synchronization accuracy, it significantly improves the system's operating efficiency and stability in complex network environments.

[0066] In the industrial IoT edge-cloud collaborative system, to adapt to the highly heterogeneous nature of field devices and the complex and ever-changing operating environment, the overall architecture is built on a container orchestration system. A centralized control plane is deployed on the cloud side for unified management of task scheduling, resource allocation, and policy distribution. Lightweight execution nodes are deployed on the edge gateways in the industrial field, enabling computing power to be pushed down to the device side, achieving low-latency response and local processing capabilities. To address the coexistence of different hardware architectures, a multi-architecture image management mechanism is introduced. Multiple versions of the same application are encapsulated using ManifestList, allowing it to adapt to different instruction set environments such as amd64 and arm64. During scheduling, node hardware architecture is used as a constraint in the computation. An architecture masking mechanism filters the scheduling space; when a candidate node does not support the corresponding instruction set, its scheduling feasibility is directly determined as unselectable, thus creating an infinite exclusion effect during computation, avoiding cross-architecture erroneous scheduling problems, and improving the stability and reliability of system operation.

[0067] A collaborative processing unit runs at the edge to achieve resource status awareness and data acquisition. The integrated resource entropy probe not only statistically analyzes the utilization rate of central processing resources but also further characterizes the volatility of the system's operating status. Specifically, it assesses computing power stability by analyzing the dispersion of CPU load within a short time window, i.e., load jitter; simultaneously, it calculates the variance of packet loss rate changes at the network layer to reflect the fluctuation characteristics of link quality; and it corrects the actual available computing power by incorporating the number of frequency reductions triggered by temperature increases during device operation. These multi-dimensional indicators are continuously updated with a sampling period of 100ms and are fused to form a quantitative expression of the current node's operational disorder. This disorder indicator not only reflects the node's current load level but also its stability and predictability, serving as a key input parameter in the scheduling model. When the disorder of a node continuously increases, its repulsive force in the overall resource distribution strengthens, thereby reducing the probability of tasks being scheduled to that node, achieving automatic avoidance of unstable nodes, and thus improving the overall system's scheduling efficiency and operational reliability.

[0068] In the industrial IoT edge-cloud collaborative system, to address the issue of reliable data synchronization in scenarios involving weak networks, network outages, and recovery, a deeply customized DeviceShadow protocol based on the MQTT communication mechanism is adopted. This protocol unifies the design of data caching, difference location, and incremental retransmission processes. The overall mechanism revolves around offline cache fragmentation, precise difference synchronization, and reliable retransmission confirmation, thereby effectively reducing network load while ensuring data consistency.

[0069] During the offline phase, the edge device does not directly upload the collected data. Instead, it writes it to the Write-Ahead Log (WAL) of the local time-series database for persistent storage. The data is managed by sharding according to fixed time windows, such as dividing it into multiple TimeChunks in 5-minute units. Each time slice is independently cached and a corresponding sub-hash value is generated. This sharding method allows data to be restored gradually in blocks after network recovery, avoiding the bandwidth impact of transmitting the entire data at once, and also providing a granular basis for subsequent difference comparisons.

[0070] After network recovery, the communication protocol implements a phased data synchronization process by defining multiple message types. The overall message structure is as follows: Table 1: Message Structure Table of Edge-Cloud Twin Data Synchronization Protocol The above messages together constitute a complete data synchronization link. The meaning and function of each field are as follows: First, `msg_type` is used to distinguish different types of messages, such as root hash reporting, difference requests, or retransmission acknowledgments, and is the basic identifier for protocol parsing. `site_id` is used to identify the site to which the data belongs, such as an industrial park or a specific production line, enabling data isolation in multi-site environments. `device_id` is used to uniquely identify a device or gateway object and is the core index for data attribution. `shadow_type` is used to distinguish different types of shadow data, such as configuration shadows or status shadows, thereby supporting parallel processing across multiple service domains.

[0071] During the consistency verification phase, root_hash serves as a summary of the entire Merkle tree, used to quickly determine whether the data between the two parties is consistent; version is used to identify the current data version, ensuring the sequentiality of the synchronization process; window_id is used to identify the time shard number, corresponding to the offline caching mechanism; leaf_count is used to record the number of leaf nodes, facilitating the verification of data scale; trace_id is used to track all messages in a complete synchronization process, ensuring the traceability of the link; and retry_flag is used to identify whether a message belongs to the retransmission or recovery phase.

[0072] When the root hashes are inconsistent, a precise location process is initiated through a difference path request message. Here, local_root_hash and remote_root_hash represent the root digests of the local and remote ends, respectively, and are used for difference comparison; request_level indicates the tree level of the current comparison; node_path is used to locate the specific node path; base_version serves as the base version for this comparison; and trace_id continues to be used for link association.

[0073] After the difference location is completed, the specific data transmission is carried out through the leaf node returned message. The leaf_key is used to identify the specific data block, the leaf_hash is used to verify the data integrity, the payload_codec is used to describe the data encoding method, such as JSON or compressed format, the payload_length is used to identify the data length, the payload_body is the actual transmitted content, and the version is used to identify the version information of the data block.

[0074] After the data retransmission is completed, closed-loop control is achieved through a retransmission confirmation message. `batch_id` identifies a single retransmission batch; `start_seq` and `end_seq` represent the start and end sequence numbers of the batch, respectively, to ensure sequential consistency; `ack_version` indicates the version confirmed by the receiver; `dedup_token` is used for deduplication verification to avoid duplicate writing; `need_retry` indicates whether further retransmission is needed; and `trace_id` is used to associate the entire synchronization transaction.

[0075] The core of the overall mechanism lies in transforming traditional full synchronization into a highly efficient incremental synchronization process through a method of rapid root hash determination, layer-by-layer location of differential paths, precise transmission of leaf nodes, and closed-loop control with batch confirmation. In weak network environments, synchronization can be completed by transmitting only the changed data blocks, significantly reducing bandwidth consumption. In network outage recovery scenarios, time window sharding and batch control enable orderly data recovery and consistency verification, thereby ensuring that the system still has stable data synchronization capabilities under complex network conditions.

[0076] In a typical industrial IoT edge-cloud collaborative verification environment, the system construction fully considers the typical characteristics of real industrial sites, such as heterogeneous equipment, unstable networks, and large data scale. The edge-side deployment environment consists of an ARM64 architecture industrial gateway and an x86 architecture industrial control computer. The two differ significantly in computing power, instruction set architecture, and resource scale to simulate the mixed deployment of multiple types of equipment in real industrial scenarios. The cloud-side configuration is a virtual computing node with an 8-core CPU and 16GB of memory resources, used to handle high-load computing tasks and centralized scheduling logic. The entire system connects to approximately 10,000 industrial data points, of which configuration data is about 30MB in size, mainly including equipment parameters, control strategies, and static configuration items; status data is continuously updated at a 1-second cycle, forming a high-frequency data stream, which places high demands on the system's data processing and synchronization capabilities.

[0077] In terms of network conditions, a hybrid 4G and 5G access method was used to construct the communication link, simulating a common wireless transmission environment in industrial settings. The available bandwidth of the link dynamically fluctuated between 4Mbps and 12Mbps, while the round-trip time (RTT) varied between 60ms and 150ms, exhibiting typical high latency and jitter characteristics. Under these conditions, traditional synchronization and scheduling mechanisms that rely on a stable network environment often struggle to maintain good performance.

[0078] Under the aforementioned unified testing criteria, static threshold scheduling and full JSON synchronization are introduced as comparative solutions. In this comparison, task scheduling is mainly based on static resource thresholds such as CPU and memory, lacking dynamic awareness of network status and computing power fluctuations; data synchronization uses full JSON data overwrite or log transmission, requiring the transmission of the complete data structure for each update. In configuration change scenarios, this method repeatedly transmits a large amount of unchanged data, resulting in low bandwidth utilization and potential congestion under network constraints.

[0079] In comparison, the optimized scheme demonstrates significant advantages under the same test conditions. When configuration data changes, the hash-based difference detection mechanism only transmits the changed data blocks, reducing the overall synchronization traffic to about one-tenth of the comparative scheme, effectively alleviating bandwidth pressure. Simultaneously, by leveraging a hierarchical structure and a step-by-step comparison of hash digests, difference path location can be completed within milliseconds, significantly shortening the data consistency verification process.

[0080] In terms of computation scheduling, when the load on edge nodes continues to increase or the network condition deteriorates, a dynamic cost evaluation mechanism adjusts the task execution location. Control tasks with strict latency requirements are prioritized for execution at the edge to ensure real-time response and stability. Analysis tasks that are interruptible or not sensitive to latency are dynamically migrated to the cloud for execution based on the current resource status, thus achieving a rational allocation of computing resources. This strategy not only avoids latency issues caused by improper scheduling of critical tasks but also improves the overall system throughput and resource utilization.

[0081] In summary, in industrial environments with large-scale equipment, complex network conditions, and diverse task types, this solution effectively addresses the shortcomings of traditional methods in terms of performance and efficiency by combining dynamic scheduling with incremental synchronization mechanisms. It reduces communication overhead while improving the stability and reliability of system operation.

[0082] This invention constructs a dynamic scheduling mechanism based on joint awareness of computing power and network status, enabling adaptive adjustment of task execution location according to the real-time operating status of nodes in complex industrial environments. When edge resources are limited or network status fluctuates, it can prioritize the stable operation of latency-sensitive tasks while rationally guiding computing load to the cloud, thereby improving overall resource utilization efficiency and reducing task execution latency. This allows the system to maintain good operational continuity and responsiveness even under heterogeneous multi-device conditions.

[0083] This invention, by introducing a hierarchical hash structure and a difference location mechanism, transforms data synchronization from full transmission to incremental transmission, enabling precise synchronization of only changed data in weak network and network outage recovery scenarios. It quickly determines consistency through hash digests and combines this with conflict handling rules to complete data merging, effectively reducing network transmission pressure and improving synchronization efficiency, making data consistency maintenance in large-scale data scenarios more efficient and reliable.

[0084] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception, characterized in that, Includes the following steps: Collect parameters such as central processing resource usage, available computing power frequency, network round-trip latency, and link throughput of edge nodes, and construct a multi-dimensional resource parameter set reflecting the node's operating status by combining historical data prediction results; The execution cost function of the task on different nodes is calculated based on the multi-dimensional resource parameter set. The tasks are classified according to the task latency constraints. The scheduling position of the task between edge nodes or cloud nodes is determined by combining the execution cost function and the task classification result. When the scheduling location changes, the running context of the corresponding task is frozen and a state snapshot is generated. The state snapshot is then migrated to the node corresponding to the scheduling location and restored, while maintaining the continuity of network connection status during task execution. For the twin data generated during task execution, it is divided into layers according to device identifier, business attributes and data fields, and the corresponding hash value is calculated based on the data of each layer to construct a multi-layer hash structure; The root hash value of the multi-level hash structure is compared. When the root hash values ​​are inconsistent, the difference data is located layer by layer and only the difference data is transmitted. At the same time, the corresponding conflict determination rules are executed according to the data attribute type to complete the data consistency synchronization.

2. The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception as described in claim 1, characterized in that, To address the problem of characterizing dynamic fluctuations in node operational status, a multi-dimensional parameter fusion processing logic is introduced, with the following steps: Data on central processing resource usage, memory usage, and network round-trip latency are collected according to a preset sampling period and then constructed as continuous time series data in chronological order. The historical data of link throughput is processed using a sliding window, and short-term forecast values ​​are calculated using the exponential moving average algorithm. The system performs dispersion analysis on the time series of central processing resource usage to calculate load jitter index, and simultaneously calculates the variance of network packet loss rate changes to generate link fluctuation index. Time series data, throughput forecasts, and various volatility indicators are uniformly normalized and combined to form a multi-dimensional resource parameter set.

3. The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception as described in claim 2, characterized in that, To establish a multi-factor coupled calculation mechanism that matches task execution cost assessment with scheduling decisions, the steps are as follows: Read the data scale information involved in the task processing process and the processing cycle data in the historical execution records, and construct a task feature set; The time required for data transmission is calculated based on the link throughput capacity, and the task execution time is calculated based on the frequency of available computing power of the node. Tasks are divided into different levels according to task latency constraints, and corresponding weight coefficients are assigned to each level. The execution cost function is obtained by weighted summation of the transmission time term and the execution time term, and the scheduling position is output accordingly.

4. The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception as described in claim 3, characterized in that, To address the stability control requirements for task migration triggering timing, a persistence determination constraint mechanism is introduced, with the following steps: The usage of central processing resources is continuously sampled and the change curves within multiple sampling periods are recorded. Statistical analysis of network egress queue depth and round-trip latency is performed, and a link status change sequence is generated. It determines whether the central processing resource usage continuously exceeds a preset threshold, and detects the duration of continuous abnormal link status. When both abnormal resource usage and abnormal link status meet the persistence condition, a migration trigger signal is generated and a scheduling control command is output.

5. The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception as described in claim 4, characterized in that, The curve of central processing resource usage change is smoothed by sliding window to form a stable trend sequence. At the same time, the fluctuation amplitude of the link status change sequence is filtered to obtain the effective state interval. The stable trend sequence and the effective state interval are combined for joint matching and judgment to generate a migration trigger signal and output scheduling control instructions.

6. The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception as described in claim 4, characterized in that, To address the requirement of maintaining the integrity of the running state during task migration, a context freezing and snapshot generation method is designed, with the following steps: The user-mode checkpoint mechanism is invoked to obtain the process tree information and execution context of the running task; Serialize the memory page data, file descriptors, and thread states involved in the process execution to generate state data; Capture and record network connection information, including communication endpoints, connection status, and buffer data; The execution context data, memory data, and network connection status are integrated to generate migration snapshot data.

7. The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception as described in claim 6, characterized in that, To ensure business continuity after cross-node migration, a mechanism for operational recovery and communication maintenance is constructed, with the following steps: The migration snapshot data is transmitted to the target node over the network and data integrity is verified. Perform deserialization on the snapshot data and restore the process execution context and memory mapping structure; Identify existing communication connections and establish corresponding forwarding rules and connection mapping tables; The network forwarding table is updated and communication path redirection is completed through kernel-mode data processing technology.

8. The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception as described in claim 7, characterized in that, To meet the needs of structured modeling of twin data, a hierarchical hash construction and digest generation logic is designed, with the following steps: Group the twin data according to the device identifier and construct device-level data nodes; The data in each device node is divided into a telemetry data set and a configuration data set according to business attributes; Extract the field identifier, data content, and time information from each data field to construct a leaf node data unit; Perform hash calculations on the leaf nodes and aggregate them level by level according to the hierarchical relationship to generate a multi-level hash structure.

9. The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception as described in claim 8, characterized in that, The leaf node hash value is formed by concatenating the field identifier, field data content and corresponding time information to form the input data, and a unified hash algorithm is used to generate a digest value. The internal nodes generate the upper-level node digest by combining the child node hash values ​​in a fixed order, forming a bottom-up consistency verification structure.

10. The method for dynamic scheduling and twin data consistency synchronization of industrial IoT edge-cloud collaboration based on heterogeneous computing power perception as described in claim 8, characterized in that, To address the issues of difference identification and conflict handling during data synchronization, an incremental synchronization and consistency control strategy is constructed, with the following steps: Exchange the root node digests of the multi-level hash structure and perform a consistency check; When there are differences in the root node digest, the hash values ​​of each node are compared level by level to locate the difference path; Extract the corresponding leaf node data based on the difference path and classify the data type according to data attributes; Apply the corresponding conflict determination rules to different types of data and complete the transmission of the difference data.

Images