Data acquisition and remote logic control method for multi-protocol equipment

Abstract

The invention discloses a data acquisition and remote logic control method for multi-protocol equipment, which comprises the following steps of: packaging data processing or control logic into a logic operator, deploying in a cloud edge execution environment, and simultaneously, sensing a network state between node computing resources and nodes in real time through a kernel probe and active detection, based on a real-time and predicted system state, through a comprehensive cost including an execution cost, a communication cost and a service quality risk cost, a dynamic migration decision algorithm is used to determine whether the system needs to be migrated and where the system is migrated to. When migration is needed, a non-interruption three-stage protocol is adopted, and the logic operator and the complete state of the logic operator are seamlessly migrated to a target node from a source node. By endowing the dynamic migration capability to the service logic, global intelligent scheduling and load balancing of computing resources are realized, the elasticity, robustness and operation and maintenance automation level of the system are improved, and meanwhile, the continuity of data processing is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of data processing and remote logic control technology for equipment, and in particular to a method for data acquisition and remote logic control of multi-protocol equipment. Background Technology

[0002] With the deep integration of the Internet of Things (IoT), the Industrial Internet, and Industry 4.0, the connection of massive heterogeneous devices to networks has become the norm. These devices typically come from different manufacturers, support a wide variety of industrial communication protocols (such as Modbus, OPC UA, EtherCAT, Profinet, etc.), and have vastly different data formats and interaction sequences. To achieve unified management and control, a typical technical solution currently adopts a cloud-edge collaborative architecture: edge computing nodes are deployed near the data source, responsible for parsing multiple protocols, initial data cleaning, and local control tasks with high real-time requirements; while the cloud platform, with its powerful computing and storage capabilities, undertakes the centralized management of massive data aggregation, complex data analysis, deep model training, and global business logic. This static division of responsibilities can effectively improve work efficiency in specific scenarios.

[0003] However, the shortcomings of the aforementioned technical solutions become increasingly apparent when facing increasingly complex and dynamic industrial and IoT application scenarios. First, multi-protocol parsing and control logic are typically tightly coupled with the underlying hardware or firmware. In existing edge gateways, data acquisition code for specific protocols (such as Modbus) is often compiled into fixed applications or firmware. This means that when a device with an unknown protocol is added to the field, or when the acquisition frequency and control algorithm of a certain protocol need to be adjusted, it often requires a complete firmware upgrade or application redeployment of the edge nodes, making it difficult to flexibly separate and independently schedule logic for a single protocol. Second, the fixed allocation of resources leads to efficiency bottlenecks. Because protocol parsing and logic control tasks are "locked" onto specific edge nodes, these nodes may remain in a state of redundant computing power for extended periods due to business downturns, resulting in wasted hardware resources. Alternatively, during business peaks (such as when a large amount of high-frequency protocol data is concurrently processed), insufficient computing power can turn them into performance bottlenecks, leading to data processing delays or even data loss. Furthermore, the system's strong dependence on network connectivity limits its reliability and response speed. Especially when business logic requiring high real-time closed-loop control is executed in the cloud, network latency, jitter, or interruptions between the edge and the cloud directly affect the timeliness of control commands, failing to meet millisecond-level industrial control requirements. Furthermore, existing solutions often exhibit complex and cumbersome operation and maintenance models. Faced with hundreds or thousands of different device protocol configurations, logic updates to a large number of distributed edge nodes typically require manual intervention or individual firmware upgrades, a tedious, time-consuming, and high-risk process that severely restricts business agility and resilience. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides a method for data acquisition and remote logic control of multi-protocol devices to solve the problems of fixed functional division, low resource utilization, strong network dependence, and complex operation and maintenance in the cloud-edge collaborative architecture mentioned in the background art.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a method for data acquisition and remote logic control of a multi-protocol device, comprising: The preset data processing or control logic is encapsulated into a logic operator that carries state information and can be independently migrated, and the logic operator is deployed in the execution environment consisting of cloud platform and edge node; Real-time perception of the computing resource status of each node in the execution environment and the network communication status between the nodes; Based on the computing resource status and network communication status, and according to a preset dynamic migration decision algorithm, it is determined whether a logical operator needs to be migrated from a source node to a target node. When the determination result indicates that migration is required, the dynamic migration process of the logical operator is executed, and the logical operator, along with its state information, is migrated from the source node to the target node without interruption, while the data flow path related to the logical operator is updated synchronously.

[0007] As a preferred embodiment of the data acquisition and remote logic control method for multi-protocol devices described in this invention, the method further includes: encapsulating the preset data processing or control logic into a logic operator carrying state information and capable of independent migration, and deploying the logic operator in an execution environment composed of a cloud platform and edge nodes. A multi-dimensional resource profile is constructed for each logical operator. The resource profile is used to quantify and characterize the expected consumption of computing resources by the logical operator under different data processing loads.

[0008] As a preferred embodiment of the data acquisition and remote logic control method for multi-protocol devices described in this invention, the real-time sensing includes: At the operating system kernel level of the edge node, the computational resource consumption of the logical operators running on it is monitored through injected kernel probe programs.

[0009] As a preferred embodiment of the data acquisition and remote logic control method for multi-protocol devices described in this invention, the real-time sensing further includes: By periodically sending active probe packets between nodes, network latency, jitter, and available bandwidth between edge nodes and the cloud platform, as well as between edge nodes themselves, are measured.

[0010] As a preferred embodiment of the data acquisition and remote logic control method for multi-protocol devices described in this invention, the dynamic migration decision algorithm includes: Based on historical and current computing resource status and network communication status, a time series prediction model is used to generate predicted system status values ​​for a future preset time window. Based on the predicted system state, a comprehensive cost is calculated for the current deployment scheme where the logical operator is retained at the source node and the candidate migration scheme for migrating it to the target node.

[0011] In a preferred embodiment of the data acquisition and remote logic control method for multi-protocol devices described in this invention, the overall cost is calculated as follows: The expected execution cost of the logical operator on the specified node, the expected communication cost between the logical operators due to cross-node deployment, and the service quality risk cost that the logical operator may face when deployed on the specified node are weighted and summed with preset weights to obtain the final comprehensive cost.

[0012] As a preferred embodiment of the data acquisition and remote logic control method for multi-protocol devices described in this invention, the step of determining whether migration is required includes: Calculate the expected cost savings of the candidate migration scheme relative to the current deployment scheme; The migration operation is only determined to be performed if the expected cost savings are greater than the sum of the one-time migration overhead required to perform the migration operation itself and a dynamic decision threshold set to maintain system stability.

[0013] As a preferred embodiment of the data acquisition and remote logic control method for multi-protocol devices described in this invention, the dynamic migration process includes the following stages: In the first stage, a running instance of the logical operator is pre-created on the target node, and the state information of the logical operator on the source node is synchronized to the running instance in an incremental and continuous manner. In the second stage, the logical operator on the origin node is briefly suspended at a preset synchronization point to transmit the final remaining state information to the target node and activate the running instance on the target node. In the third stage, all input data streams pointing to the logical operators on the source node are redirected to the activated running instance on the target node through a coordination mechanism.

[0014] As a preferred embodiment of the data acquisition and remote logic control method for multi-protocol devices described in this invention, the first stage includes: The operating system's copy-on-write mechanism is used to track changes in the state information of logical operators on the source node, and only the modified state information is asynchronously transmitted to the target node to reduce the impact of the synchronization process on the execution performance of the source node's logical operators.

[0015] As a preferred embodiment of the data acquisition and remote logic control method for multi-protocol devices described in this invention, the third stage employs a two-stage submission coordination mechanism, including: The mechanism first sends a ready-to-switch instruction to all upstream data sources that provide data for the logical operator. After receiving ready confirmation from all data sources, it then broadcasts the switch instruction to ensure that all data sources synchronously switch their data streams to the target node at the same logical moment.

[0016] Compared with existing technologies, the beneficial effects of this solution are: 1. By encapsulating business logic into portable logic operators and making comprehensive cost decisions based on real-time perception and future state prediction, computing tasks can be intelligently migrated from resource-constrained nodes to resource-rich nodes, thereby achieving global load balancing, eliminating performance bottlenecks, avoiding idle computing power on edge nodes during business downturns and performance crashes during business peaks, and improving the overall utilization efficiency of hardware resources.

[0017] 2. When the latency or jitter of the edge-cloud network exceeds the acceptable range, control logic operators with high real-time requirements can be automatically migrated to edge nodes closer to the device for execution. This eliminates the dependence on unstable wide area networks, ensures that millisecond-level industrial control commands can be issued stably, and enhances the reliability of the entire system and the ability to guarantee the quality of critical business services.

[0018] 3. By encapsulating standardized logic operators, business logic is decoupled from the underlying hardware, simplifying the updating, rollback, or deployment of logic into remote and dynamic scheduling of logic operator packages. Thus, without manual intervention, the system can automatically complete the replacement and migration of old and new operators, enabling rapid iteration and elastic scaling of business operations and improving the operational efficiency and agility of large-scale distributed cloud-edge systems.

[0019] 4. By utilizing a copy-on-write mechanism for efficient incremental state synchronization and employing a two-phase commit protocol to achieve atomic redirection of data streams, the business downtime during logical operator migration can be compressed to the millisecond level, making it almost transparent to upper-layer applications. This ensures that while the system is optimizing itself, the continuity of data acquisition and control tasks remains unaffected, solving the technical challenge of implementing dynamic migration strategies in high-availability scenarios such as industry. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart illustrating the overall process of a data acquisition and remote logic control method for a multi-protocol device according to an embodiment of the present invention. Detailed Implementation

[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0022] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0023] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0024] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0025] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0026] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0027] Example 1 Reference Figure 1 This is the first embodiment of the present invention, which provides a method for data acquisition and remote logic control of a multi-protocol device, including: S1. Encapsulate the preset data processing or control logic into logic operators that carry state information and can be independently migrated, and deploy the logic operators in the execution environment consisting of the cloud platform and edge nodes.

[0028] Specifically, in a distributed system comprised of a cloud platform and various edge nodes, basic management components are deployed to build a standardized execution environment that can operate heterogeneously between the cloud and the edge. This process includes: Deploy a lightweight monitoring agent on each edge node and cloud platform node. This monitoring agent is responsible for performing resource and network status awareness tasks and periodically reporting data upwards.

[0029] Deploy a centralized migration coordinator on a cloud platform or a highly available master edge node. This coordinator is responsible for aggregating monitoring data from all nodes, executing dynamic migration decision algorithms, and initiating and scheduling the entire logical operator migration process.

[0030] It should be noted that this step aims to transform the upper-level business requirements into atomic functional units, i.e., logical operators, that can flow and be executed in this execution environment.

[0031] It is important to emphasize that in this embodiment, any data processing logic, such as a linear transformation function for temperature correction or an AI inference model for device fault prediction, is encapsulated as an independent logic execution unit. This encapsulation process aims for lightweight design and cross-platform compatibility to adapt to the resource-constrained characteristics of edge nodes.

[0032] Specifically, in this embodiment, WebAssembly (Wasm) is preferably used as the core execution carrier for the logic operators. Considering the characteristics of multi-protocol devices, the parsing code libraries for different industrial protocols (such as Modbus TCP, OPC UA, MQTT, etc.) are compiled into independent Wasm modules. For example, for a temperature control device supporting the Modbus protocol, the device's polling instruction generation logic and register data parsing logic are encapsulated into a Modbus acquisition logic operator; the device's PID temperature control algorithm is encapsulated into a temperature control-calculation logic operator. It should be explained that Wasm is chosen because of its platform independence, near-native execution efficiency, and robust security sandbox mechanism. In terms of business logic (e.g., written in high-performance languages ​​such as Rust and C++), Wasm can be compiled into Wasm bytecode modules, ensuring that the same logic operator can run in a predictable high-performance mode on the cloud (typically x86 architecture) and different types of edge nodes (e.g., ARM architecture) without any modification.

[0033] Specifically, each logical operator is packaged into a compressed archive file, and the internal structure of this file is standardized. The packaged content includes: The Wasm module (logic.wasm): contains compiled bytecode containing core business logic.

[0034] Metadata description file (manifest.json): A JSON file that defines the static properties of the logical operator in detail, including: globally unique operator ID, version number, input / output port definition (including data type and semantic tags), dependencies between operators (used to build a directed acyclic graph DAG for logical processing), and quality of service (QoS) requirements (e.g., maximum tolerable execution latency or data processing priority).

[0035] The state schema definition file (state.proto): This file defines the structure of all states that need to be maintained within the operator (such as intermediate calculation results of the sliding window, counters, model parameters, etc.) to achieve migrations that "carry state information." It is defined using Protocol Buffers. Compared to JSON, Protocol Buffers provides strong type constraints, efficient binary serialization / deserialization capabilities, and backward-compatible schema evolution support, which is crucial for achieving fast, low-overhead state transfer during migrations.

[0036] It should be noted that, through the above encapsulation operations, any complex business flow can be deconstructed into standard, self-describing logical operators that can be independently deployed and migrated.

[0037] Furthermore, an accurate quantization model is established for each logical operator, enabling the subsequent migration decision algorithm to "predict" the resource consumption of each logical operator under different workloads.

[0038] Specifically, firstly, in a standardized testing environment, a systematic performance analysis is performed on each version of the logic operator. It's important to note that we do not use traditional single-point measurements; instead, we construct a multivariate function model of data input rate and data complexity to characterize its resource consumption. Then, using a load generator, different combinations of data input rates and data complexity are used to drive the logic operator's operation, and high-precision monitoring tools are used to record underlying metrics such as CPU utilization and memory usage. Based on the large amount of collected data, a multivariate nonlinear regression method is used to fit its resource consumption model. In one feasible implementation of this embodiment, the resource consumption model for CPU utilization using the multivariate nonlinear regression method can be expressed as: in, This represents the predicted CPU utilization. This represents the data input rate (e.g., messages per second). It is a dimensionless data complexity factor, which can be the average size of the data packet, the number of iterations of the calculation, or other indicators that can characterize the difficulty of a single processing step. These are model coefficients obtained through nonlinear regression methods.

[0039] It should be noted that resource consumption fitted using the multivariate nonlinear regression method, compared to traditional linear models, can more accurately capture the nonlinear effects of resource consumption as load increases (such as decreased cache hit rate and increased scheduling overhead), thus providing more accurate predictions. Furthermore, these model coefficients, along with key parameters such as memory consumption, constitute the "resource profile" of this logical operator and are stored in its metadata.

[0040] Furthermore, resource consumption models are typically implemented offline, which fails to account for dynamic factors such as resource contention and system jitter in real-world operating environments. Therefore, in this invention, a monitoring agent deployed on each node continuously observes the actual resource consumption of the logical operators. This is achieved by employing a Kalman filter with a forgetting factor to correct the offline resource consumption model online.

[0041] Specifically, at each time step, a Kalman filter treats the predicted value from the offline resource consumption model as the system's "predicted state" and the real-time monitored resource consumption value as a noisy "observation." Then, the Kalman filter recursively calculates and merges the "predicted state" and the "observation" to generate a smoother posterior estimate of the current actual resource consumption, ensuring that the logic operator's "resource profile" can dynamically adapt to the constantly changing operating environment.

[0042] S2. Real-time perception of the computing resource status of each node in the execution environment and the network communication status between nodes.

[0043] It should be noted that the core of this step is to obtain the overall operational status of the entire cloud-edge collaborative system in a low-overhead, high-precision, and high-frequency manner. This operational status includes not only the internal load of each computing node (cloud platform or edge node), but also the health status of the network transmission connecting these computing nodes.

[0044] Furthermore, in order to achieve accurate measurement of the resource consumption of logical operators, while minimizing the performance impact of the monitoring behavior itself on the execution of business logic, the present invention abandons the traditional monitoring method based on user-mode polling (such as periodically executing the ps or top commands), and instead performs monitoring at the operating system kernel level of the edge node by injecting a lightweight kernel probe program.

[0045] Specifically, in this embodiment, eBPF (extended Berkeley Packet Filter) technology is preferably used in Linux-based edge node operating systems. eBPF technology allows for the safe and efficient execution of verified bytecode in kernel space without modifying the kernel source code or loading complete kernel modules. We pre-write eBPF programs and attach them to key kernel functions or tracepoints related to process scheduling, memory management, and I / O operations. In this embodiment, two attachment methods are provided: the first method attaches to scheduler-related tracepoints (such as the `sched_switch` function), which can accurately accumulate the CPU time slices occupied by a logical operator (corresponding to one or more processes / threads) at each context switch. The second method attaches to memory page fault handling functions, which can track changes in the resident memory set (RSS) and virtual memory size (VSS) of logical operators in real time.

[0046] Specifically, the raw data collected by the eBPF program in kernel space is asynchronously transmitted to the monitoring agent running in user space using a highly efficient shared memory mechanism, namely eBPF Maps (such as BPF_MAP_TYPE_PERF_EVENT_ARRAY or BPF_MAP_TYPE_RINGBUF, where BPF_MAP_TYPE_PERF_EVENT_ARRAY is a special mapping type in eBPF used to achieve efficient data transfer from kernel space to user space; similarly, BPF_MAP_TYPE_RINGBUF is used to efficiently transfer data between kernel space (BPF program) and user space). This process uses zero-copy to avoid the overhead of frequent data copying and system calls between kernel space and user space, ensuring low latency and low intrusion in monitoring. Then, the monitoring agent aggregates the collected raw data (e.g., aggregating CPU time by second) and associates it with the unique ID of the logical operator, ultimately forming structured resource status time-series data.

[0047] Furthermore, to quantify the network connection quality between nodes, the present invention employs a method of periodically sending active probe packets to perform end-to-end measurements of key network metrics. In this embodiment, the selected key network metrics include: network latency, jitter, and available bandwidth.

[0048] Furthermore, during end-to-end measurement of network metrics, lightweight UDP probe packets are periodically (e.g., every second) sent between monitoring agents on each node. It is important to emphasize that, for accurate calculation of One-Way Delay (OWD), the system clocks of all nodes must be synchronized with high precision via NTP (Network Time Protocol) or PTP (Precision Time Protocol).

[0049] Specifically, for network latency, the probe packet payload contains a precise timestamp of its transmission time. And the recipient records a local timestamp when it receives the packet. One-way delay is In this process, round-trip time (RTT) can also be introduced, which is measured in a request-response pattern without requiring the system clock synchronization mentioned above, and can serve as an auxiliary or backup indicator.

[0050] Specifically, jitter is a metric that characterizes the degree of variation in network latency. In this embodiment, the Real-time Transport Protocol (RTP) and Real-time Control Protocol (RTCP) from IETF RFC 3550 are used to calculate the transmission latency difference between two consecutive probe packets. Assume the... The and the first The one-way delays of each packet are respectively and Then it will shake instantly. It can be represented as: In addition, to obtain a smooth jitter value that reflects recent network conditions, the system maintains an exponentially weighted moving average (EWMA) jitter value. This value will be used as an important input for service quality risk assessment. Specifically, this jitter value... It can be represented as: in, It is a smoothing factor, taking the value of (For example, 0.1), whose value determines the weight of historical data and current measurements.

[0051] It should be noted that estimating available bandwidth is more practical than measuring link capacity, because available bandwidth reflects the remaining capacity on the current path available for new traffic. In measuring the available bandwidth metric, the present invention employs a non-intrusive method called "Packet Pair Dispersion".

[0052] Specifically, the monitoring agent starts from the source node. To the target node Send two consecutively of the same size (both) A UDP probe packet (i.e., a "packet pair") is sent in pairs of packets with only a very small interval between them. When this pair of packets passes through the narrowest bottleneck link on the path, the second packet must wait for the first packet to completely pass before it can begin transmission, thus "stretching" the interval between them. The target node... This will measure the arrival time interval of the two packets. Therefore, the instantaneous capacity of the bottleneck link can be estimated using the following formula: Furthermore, to estimate available bandwidth, rather than link capacity, more complex packet sequences (such as "packet strings") and algorithms are often required. In this embodiment, by sending a series of probe packets and analyzing the distribution pattern of their arrival time intervals, interference caused by existing traffic in the network is filtered out, thereby estimating a more accurate available bandwidth.

[0053] It is important to emphasize that by employing active probing rather than passive observation, the present invention's solution can independently and stably quantify network status without relying on actual business traffic. The choice of UDP as the probing protocol is to bypass TCP's congestion control and retransmission mechanisms, thereby measuring the most basic physical characteristics of the network path.

[0054] S3. Based on the computing resource status and network communication status, and according to a preset dynamic migration decision algorithm, determine whether a logical operator needs to be migrated from a source node to a target node.

[0055] It should be noted that this step abandons the traditional reactive strategy of static thresholds and makes the optimal logic operator deployment decision through prediction and multi-dimensional cost-benefit analysis.

[0056] Furthermore, in order to avoid making short-sighted and unstable migration decisions due to instantaneous fluctuations in the system state, it is first necessary to predict the system state over a period of time in the future, aiming to shift the decision-making from "post-event remediation" to "pre-event avoidance".

[0057] Specifically, the input to the dynamic migration decision algorithm is not only the current state of computing resources and network communication in step S2, but also a preset time window for the future based on historical and current state time series data. The system's state evolution trend is predicted. Key variables for prediction include: Node load prediction: For each node Predict CPU utilization and memory usage.

[0058] Network state prediction: for node pairs Predict network latency, jitter, and available bandwidth.

[0059] Operator load prediction: For each logic operator Input data rate and data complexity Make predictions.

[0060] Furthermore, considering that workloads in industrial scenarios often exhibit both periodicity (e.g., alternating day and night shifts) and suddenness (e.g., alarm events), traditional linear models (such as ARIMA) struggle to accurately capture their complex nonlinear dynamics. Therefore, in this embodiment, a Long Short-Term Memory (LSTM) network is preferred as the time series prediction model. It should be explained that LSTM is chosen because its unique gating mechanism (input gate, forget gate, output gate) effectively learns and remembers long-term dependencies in time series data while filtering out unimportant short-term disturbances. This gives the model a significant accuracy advantage over traditional models when predicting resource and load data with complex patterns. For each variable requiring prediction, an independent LSTM model is trained. It is important to emphasize that this LSTM model is trained offline in the cloud and can be incrementally updated periodically using newly collected data to adapt to the long-term evolution of business models.

[0061] Furthermore, in obtaining future time windows After predicting the system state, the dynamic migration decision algorithm needs a unified benchmark to measure the "goodness" or "badness" of any deployment scheme. To this end, this invention constructs a comprehensive cost function. This function consists of a weighted average of three sub-costs. Assume it needs to be a logical operator. Evaluate a candidate solution: extract it from the source node Migrate to target node The algorithm will calculate separately. Stay (Current plan) and move to (Candidate solutions) in the future time window The expected total cost within.

[0062] Specifically, this comprehensive cost is expressed as follows: in, It is a logical operator Deployed on nodes The overall cost. These are preset, dimensionless weighting coefficients that satisfy... .

[0063] Specifically, in the aforementioned comprehensive costs, The expected execution cost quantifies the value of computational resources consumed by the logical operator when running on a specific node. This is achieved using the resource profile constructed in step S1 and the operator load predicted in step S3. Logical operators can be predicted. Resource needs in the future (A vector containing CPU, memory, and other requirements). Combined with nodes. Resource unit price vector (For cloud nodes, this is the actual billing price; for edge nodes, it is the equivalent cost of hardware depreciation and electricity.) The expected execution cost is calculated as follows: in, This represents the initial time. It is represented as the transpose of the resource unit price vector.

[0064] It should be noted that this cost calculation formula multiplies the future resource consumption of the operator with the "economic value" of its node, enabling a unified quantitative comparison of heterogeneous resource costs.

[0065] Specifically, in the aforementioned comprehensive costs, The expected communication cost reflects the network overhead incurred due to cross-node communication between logical operators. When logical operators... Deployed in When it is above, it is related to all its upstream logical operators. and downstream operators The communication costs all need to be calculated. Assuming logical operators... Deployed on nodes The predicted data flow rate between them is as follows: The expected communication cost can be expressed as: in, It is used to describe at time 10:00 From node arrive The network cost function for transmitting one unit of data can be a function of predicting network latency. and predicted available bandwidth In one possible implementation of this embodiment, the network cost function can be expressed as: This cost function is used to ensure that the decision-making algorithm will tend to deploy logical operators with a large amount of data interaction on nodes with high network connectivity, or even on the same node (where the communication cost is 0).

[0066] Specifically, in the aforementioned comprehensive costs, The anticipated QoS risk cost, which monetizes the risk of potential QoS violations, enables the decision-making algorithm to proactively avoid potential performance issues. This cost comprises resource overload risk and network latency risk. The anticipated QoS risk cost can be expressed as: in, It is a node At any moment The probability of resource overload occurring. This occurs when the predicted total resource demand (including existing logical operators and those to be deployed on the node) approaches or exceeds the node's capacity. At this point, the probability increases sharply. Here, we use a sigmoid function to smoothly model this probability: in, It is the predicted total node utilization, which is equal to the sum of the predicted resource consumption of all resident logic operators on the node and the expected resource consumption of the logic operators to be migrated. It is a safety threshold, and its value ranges from... (e.g., 80%) This is a control coefficient used to control the steepness of the probability curve; its value range is... . It is the probability that the end-to-end processing delay exceeds the service quality requirements of the logical operator due to network jitter or other reasons. and This is the penalty constant for the corresponding term, and its value depends on the business's requirements for stability and real-time performance. It typically ranges from [value range missing]. .

[0067] Furthermore, after calculating the combined cost of the current solution and all candidate solutions, the final step of the decision-making algorithm is to determine whether the migration is worthwhile. It is important to emphasize that an immature decision-making system may migrate frequently due to small cost differences, causing system "shaking" or "bumping," where the cost of migration itself outweighs the actual benefits. To address this issue, this invention introduces a decision-triggered mechanism based on the hysteresis effect. This mechanism includes: Calculate the expected cost savings For logical operators From the source node Migrate to candidate target node The proposed solution is expected to save the following costs: .

[0068] Assess migration costs Because the migration operation itself is not zero-cost, This is the one-time overhead required to perform a migration, which includes: the network bandwidth cost of transmitting operator state information, the CPU consumption cost of the source and target nodes during state serialization / deserialization, and the equivalent cost of potential performance degradation or service interruption during the short switching window. This migration overhead assessment can be dynamically estimated based on the state size of the logical operators and the current network conditions.

[0069] Apply dynamic decision threshold To maintain system stability, this invention introduces a dynamic decision threshold. (Also known as the hysteresis threshold). The physical meaning of this threshold is that migration is only triggered when the "net benefit" of the migration is large enough. This effectively filters out invalid migration decisions caused by prediction errors or minor environmental fluctuations.

[0070] Formation of decision rules: only when Only when these conditions are met will the system finally determine whether to execute the command. arrive The migration operation. Furthermore, to enhance stability, this threshold... It can be dynamic. For example, temporarily increasing the size of a migration shortly after it has completed. The value of is used to create a "decision cooling-off period" to prevent the logical operator from being immediately migrated back.

[0071] It should be noted that, through the above-mentioned prediction, quantification, comparison, and hysteresis judgment process, the dynamic migration decision algorithm of the present invention can accurately, stably, and proactively find the optimal deployment position of the logic operator in a complex cloud-edge environment, thereby achieving global optimization of system computing resources.

[0072] S4. When the judgment result indicates that migration is required, execute the dynamic migration process of the logical operator, migrate the logical operator along with its state information from the source node to the target node without interruption, and synchronously update the data flow path related to the logical operator.

[0073] It should be noted that the traditional "stop-replicate-start" model results in minute-level service interruptions, which is unacceptable in industrial control and high-frequency data acquisition. Therefore, the goal of this step is to determine how to migrate a logical operator with complex internal states from one physical node to another without interrupting ongoing data processing. Furthermore, in this invention, the entire dynamic migration process is scheduled by a centralized migration coordinator (which can also run on the cloud or a highly available edge master node), and it is divided into three phases.

[0074] Furthermore, the first phase involves pre-replication and incremental synchronization. The goal of this phase is to replicate the vast majority of state data to the target node in advance, while the source node's logical operators are running fully normally, thereby minimizing the service downtime required during the subsequent formal switchover.

[0075] Specifically, the migration coordinator first sends an instruction to the target node, requesting it to pull the Wasm module in advance and create a running instance of the logical operator in a "standby" state based on the metadata description file (manifest.json) of the logical operator to be migrated. It is important to note that when creating the logical operator instance, the Wasm runtime of the edge node registers the virtual address range of the linear memory occupied by the logical operator instance with the kernel probe module. This kernel probe module only enables copy-on-write and dirty page tracking mechanisms for memory pages within this registered address range, thereby achieving precise and independent capture of the state of specific logical operators, unaffected by runtime itself or memory activities of other operators. Then, in order to continuously synchronize the state changes of the source logical operator without significantly affecting its performance, this invention adopts an incremental synchronization method based on the operating system's copy-on-write (CoW) mechanism, the detailed process of which is as follows: The migration coordinator instructs the source node to take a full snapshot of the current state memory area of ​​the logical operator and transmit this snapshot data to the standby instance on the target node as its initial state. Simultaneously, the source node's operating system kernel (via the kernel module injected in this invention), in conjunction with the Wasm runtime, identifies the linear memory region corresponding to the running instance of the logical operator in the process address space and marks all memory pages within that region as read-only. When a running logical operator on the source node attempts to modify any of the read-only pages in its state memory, an operating system-level page fault interrupt is triggered. The kernel module pre-defined in this invention's scheme captures this interrupt. At this point, the kernel does not pass the error to the logical operator but instead performs a "copy-on-write" operation: the kernel first creates a copy of the "dirty page" to be written to, then places the unmodified original dirty page into a dedicated "incremental synchronization queue," and finally allows the logical operator's write operation to be completed on the new copy page. This entire process is completely transparent to the logical operator itself, and therefore its execution is almost unaffected. The significance of this operation lies in achieving real-time tracking of state changes with extremely low performance overhead, avoiding costly application-layer polling or comparison operations. Finally, a separate synchronization thread continuously retrieves dirty pages from this "incremental synchronization queue" and sends them to the pre-created instances on the target node via asynchronous network transmission. Upon receiving these incremental state updates, these created running instances apply them to their own memory space, gradually catching up with the state of the source running instance. Furthermore, this process is continuous and parallel, occurring synchronously with the normal business processing of the source running instance.

[0076] It should be noted that through the first phase of operation, the vast majority (usually more than 99%) of the state data can be migrated without stopping the source logic operator, thus laying the foundation for achieving the final "uninterrupted" switching.

[0077] Furthermore, the second phase is Suspend-Commit and Activate. This phase is the only time window in the entire migration process that requires a brief interruption of service. Its core objective is to complete the final state synchronization and switch the service entity within a controllable and extremely short time.

[0078] Specifically, the migration coordinator continuously monitors the length of the "incremental synchronization queue" or the rate of dirty page generation during the first phase. When this rate drops to a preset, sufficiently low threshold (e.g., the number of dirty pages generated per second is less than a specific value over several consecutive synchronization cycles), the system determines that the optimal "synchronization point" has been reached. It's important to explain that the significance of choosing this "synchronization point" for switching is that it ensures the amount of data transferred in the last state transfer is extremely small, thereby minimizing service interruption time (typically on the order of milliseconds). Once the synchronization point is reached, the migration coordinator immediately sends a "suspend" instruction to the logical operator runtime on the source node. Upon receiving the instruction, the logical operator runtime immediately stops processing new input data (at this point, the upstream data source begins caching data) and completes the last data unit it is currently processing. Then, after the source logical operator is suspended, the migration coordinator performs a final state synchronization, transferring the remaining, minimally sized dirty pages in the source logical operator's memory since the last synchronization, along with the final context information such as CPU register states, program counters, and (for Wasm, primarily global variables and call stack information), to the target node all at once. Finally, after receiving and applying this last part of the state, the running instance on the target node has an internal state completely consistent with the source instance's state at the moment of suspension. At this point, the migration coordinator sends an "activate" command to the target instance, which immediately transitions to an active state, ready to receive and process new data streams.

[0079] Furthermore, the third stage is the atomic redirection of data streams. This stage addresses how to ensure that all upstream data sources can accurately and synchronously switch data streams from suspended source nodes to newly activated target nodes during the migration process, avoiding data loss or duplicate processing. To solve this problem, this invention employs a robust two-phase commit (2PC) coordination mechanism.

[0080] Furthermore, in this coordination mechanism, the coordinator is defined as the central control module that executes migration decisions. Participants are defined as all upstream logical operators or external data sources that provide input data for the migration logical operator.

[0081] Specifically, first, during or after the second-phase "suspend" operation, the coordinator broadcasts a "PREPARE_SWITCH" instruction to all participants. This instruction contains the network address of the newly activated target logical operator. Upon receiving this instruction, each participant enters the "ready" state. It immediately stops sending data to the old source address and temporarily buffers newly generated data in its local output buffer. During this process, the participant verifies the reachability of the new address. If everything is normal, it replies to the coordinator with a "VOTE_COMMIT" instruction. If the switch cannot proceed for any reason (such as network failure), it replies with a "VOTE_ABORT" instruction. The coordinator then collects the replies from all participants. If the coordinator receives "VOTE_COMMIT" instructions from all participants, it confirms that the switch can proceed safely and synchronously. Simultaneously, the coordinator broadcasts a "COMMIT_SWITCH" instruction to all participants. Upon receiving the "Commit Switch" instruction, each participant performs an atomic operation: it formally changes the target address of its data output to the new target node address, and then sends all data cached during the preparation phase to the new target logical operator in a single, sequential manner. This allows for seamless redirection of the data stream. If any participant responds with an "Abort" instruction, or fails to respond within the preset timeout period, the coordinator determines the switch has failed. It broadcasts an "Abort Switch" (ABORT_SWITCH) instruction to all participants. Participants receiving this instruction clear their caches and roll back to the state of continuing to send data to the original node, or wait for a new migration instruction.

[0082] It should be noted that by introducing a two-phase commit protocol, this invention ensures that the switching of data streams is a "unanimous action" process. This guarantees that at any given time, a data packet is either sent to the old address or the new address, and there will never be an intermediate state where some sources are switched while others are not, thus achieving strong consistency and zero data loss in data processing.

[0083] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0084] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0085] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0087] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0088] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for data acquisition and remote logic control of a multi-protocol device, characterized in that, include: The preset data processing or control logic is encapsulated into a logic operator that carries state information and can be independently migrated, and the logic operator is deployed in the execution environment consisting of cloud platform and edge node; Real-time perception of the computing resource status of each node in the execution environment and the network communication status between the nodes; Based on the computing resource status and network communication status, and according to a preset dynamic migration decision algorithm, it is determined whether a logical operator needs to be migrated from a source node to a target node. When the determination result indicates that migration is required, the dynamic migration process of the logical operator is executed, and the logical operator, along with its state information, is migrated from the source node to the target node without interruption, while the data flow path related to the logical operator is updated synchronously.

2. The data acquisition and remote logic control method for multi-protocol devices as described in claim 1, characterized in that, The step of encapsulating preset data processing or control logic into a logic operator that carries state information and can be independently migrated, and deploying the logic operator in an execution environment consisting of a cloud platform and edge nodes, further includes: A multi-dimensional resource profile is constructed for each logical operator. The resource profile is used to quantify and characterize the expected consumption of computing resources by the logical operator under different data processing loads.

3. The data acquisition and remote logic control method for multi-protocol devices as described in claim 1, characterized in that, The real-time sensing includes: At the operating system kernel level of the edge node, the computational resource consumption of the logical operators running on it is monitored through injected kernel probe programs.

4. The data acquisition and remote logic control method for multi-protocol devices as described in claim 3, characterized in that, The real-time sensing also includes: By periodically sending active probe packets between nodes, network latency, jitter, and available bandwidth between edge nodes and the cloud platform, as well as between edge nodes themselves, are measured.

5. The data acquisition and remote logic control method for multi-protocol devices as described in claim 1, characterized in that, The dynamic migration decision algorithm includes: Based on historical and current computing resource status and network communication status, a time series prediction model is used to generate predicted system status values ​​for a future preset time window. Based on the predicted system state, a comprehensive cost is calculated for the current deployment scheme where the logical operator is retained at the source node and the candidate migration scheme for migrating it to the target node.

6. The data acquisition and remote logic control method for multi-protocol devices as described in claim 5, characterized in that, The calculation method for the comprehensive cost is as follows: The expected execution cost of the logical operator on the specified node, the expected communication cost between the logical operators due to cross-node deployment, and the service quality risk cost that the logical operator may face when deployed on the specified node are weighted and summed with preset weights to obtain the final comprehensive cost.

7. The data acquisition and remote logic control method for multi-protocol devices as described in claim 5, characterized in that, The step of determining whether migration is necessary includes: Calculate the expected cost savings of the candidate migration scheme relative to the current deployment scheme; The migration operation is only determined to be performed if the expected cost savings are greater than the sum of the one-time migration overhead required to perform the migration operation itself and a dynamic decision threshold set to maintain system stability.

8. The data acquisition and remote logic control method for multi-protocol devices as described in claim 1, characterized in that, The dynamic migration process includes the following stages: In the first stage, a running instance of the logical operator is pre-created on the target node, and the state information of the logical operator on the source node is synchronized to the running instance in an incremental and continuous manner. In the second stage, the logical operator on the origin node is briefly suspended at a preset synchronization point to transmit the final remaining state information to the target node and activate the running instance on the target node. In the third stage, all input data streams pointing to the logical operators on the source node are redirected to the activated running instance on the target node through a coordination mechanism.

9. The data acquisition and remote logic control method for multi-protocol devices as described in claim 8, characterized in that, The first stage includes: The operating system's copy-on-write mechanism is used to track changes in the state information of logical operators on the source node, and only the modified state information is asynchronously transmitted to the target node to reduce the impact of the synchronization process on the execution performance of the source node's logical operators.

10. The data acquisition and remote logic control method for multi-protocol devices as described in claim 8, characterized in that, The third phase employs a two-phase submission coordination mechanism, including: The mechanism first sends a ready-to-switch instruction to all upstream data sources that provide data for the logical operator. After receiving ready confirmation from all data sources, it then broadcasts the switch instruction to ensure that all data sources synchronously switch their data streams to the target node at the same logical moment.

Images