Collaborative data replica control method and system for real-time machine learning on cloud

Abstract

The application belongs to the field of distributed storage, and provides a collaborative data copy control method and system for real-time machine learning on the cloud, which evaluates each candidate edge node by using a comprehensive cost function, determines the optimal copy location with the minimum comprehensive cost function as the target, selects several adjacent nodes as backup locations, synchronizes the data copies between the cloud and the edge nodes by using the federated learning technology, detects the data copies by using the Byzantine fault tolerance mechanism, and removes the edge nodes with abnormal information synchronization; the data copy scheduling problem in a dynamic environment is abstracted as a Markov decision process for optimization and solution, and closed-loop control is performed according to the result of the optimization and solution. The application can realize cost-aware copy placement, safe and reliable cooperation in the cloud-edge-end environment, and dynamic closed-loop control, and reduce the storage and transmission overhead while reducing the delay and improving the reliability.

Description

Technical Field

[0001] This invention belongs to the field of distributed storage, specifically relating to a collaborative data replica control method and system for real-time machine learning in the cloud. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of new-generation information technology, the Industrial Internet has gradually become a crucial infrastructure for driving the digital transformation and intelligent upgrading of the manufacturing industry. The Industrial Internet, with data at its core, connects massive amounts of equipment, sensors, and production systems. Its typical characteristics are multi-source heterogeneity, massive scale, and high real-time requirements. In this environment, massive amounts of industrial data need to be efficiently stored, transmitted, and computed in the cloud, edge nodes, and terminal devices (cloud-edge-device collaborative architecture) to support complex real-time machine learning and intelligent decision-making tasks.

[0004] In traditional cloud computing, data is typically stored centrally in remote data centers and processed uniformly in the cloud. However, real-time control and prediction in industrial scenarios often have extremely high requirements for latency and reliability. Relying solely on the cloud not only increases data transmission costs but can also cause network congestion and response delays. For example, in smart manufacturing scenarios, vibration sensors on production line equipment collect operational status data at kHz frequencies. If all data needs to be transmitted to the cloud for processing, network transmission latency alone could reach tens of milliseconds, far exceeding the millisecond-level response requirements of industrial control. Therefore, edge computing has gradually emerged. Its core idea is to perform some computation and storage on edge nodes (such as industrial gateways and edge servers) close to the data source, thereby reducing network transmission overhead and latency.

[0005] Furthermore, edge intelligence, by deploying machine learning and artificial intelligence models at edge nodes, enables rapid analysis and intelligent decision-making of local data, becoming a key supporting technology for the Industrial Internet. Typical edge intelligence applications include real-time anomaly detection of production line equipment and instant optimization of process parameters, all of which rely on low-latency data access and efficient data copy management.

[0006] However, replica management in a cloud-edge-device collaborative environment still faces many challenges. On the one hand, insufficient dynamic adaptability leads to low resource efficiency: the placement and migration of data replicas are often based on static rules, which cannot dynamically respond to real-time changes in industrial tasks. When edge nodes suddenly become overloaded or lack resources, the system cannot optimize replica distribution in time, causing critical data to be forced to migrate to high-latency or low-performance nodes, resulting in increased storage costs, increased access latency, and even affecting production continuity and security.

[0007] On the other hand, the reliability and security of cross-domain collaboration face severe challenges. Cloud-edge-device nodes in the Industrial Internet are deployed in heterogeneous and dynamically unstable network environments. Edge node hardware reliability is low, and there is a risk of untrusted or semi-trusted nodes, making it difficult to guarantee data consistency and reliability.

[0008] Finally, the lack of dynamic feedback and adaptive optimization capabilities means that the existing replica management strategies, which rely on periodic offline optimization or manual intervention, cannot respond quickly to real-time status changes. When the storage resources of an edge node are suddenly exhausted or network latency suddenly increases, traditional strategies may take several minutes or even hours to detect and migrate replicas, which seriously affects industrial production efficiency and product quality. Summary of the Invention

[0009] To address the aforementioned issues, this invention proposes a collaborative data replica control method and system for real-time machine learning in the cloud. This invention enables cost-aware replica placement, secure and reliable collaboration in cloud-edge-device environments, and dynamic closed-loop control, thereby reducing storage and transmission overhead while ensuring low latency and high reliability. This supports real-time data processing and intelligent applications in industrial internet and edge intelligence scenarios.

[0010] According to some embodiments, the present invention adopts the following technical solution:

[0011] A collaborative data replication control method for real-time machine learning in the cloud includes the following steps:

[0012] For data replicas of data blocks, a comprehensive cost function is used to evaluate each candidate edge node. The optimal replica location is determined with the goal of minimizing the comprehensive cost function, and several neighboring nodes are selected as backup locations. The comprehensive cost function is a weighted fusion of static cost, dynamic cost, and business constraints.

[0013] The system acquires real-time parameters of each edge node. When the real-time parameters of the edge node meet the preset triggering conditions, it redistributes data replicas, re-evaluates the comprehensive cost function, determines the optimal replica location, and migrates the corresponding replicas.

[0014] Federated learning technology is used to synchronize data copies between the cloud and edge nodes. The cloud model is shared on each edge node, and the cloud periodically broadcasts the latest parameters of the global model to each edge node. Each edge node uses local data to train the model and uploads the updated model parameters to the cloud for aggregation. If the data uploaded by an edge node deviates from the expected model parameters by more than a set range, the Byzantine fault tolerance mechanism is used to detect data copies and remove edge nodes with abnormal information synchronization.

[0015] The data replica scheduling problem in a dynamic environment is abstracted into a Markov decision process. It comprehensively considers resource utilization, load balancing, access latency, and replica availability to optimize the solution. Based on the optimization results, the optimal replica management strategy is determined. The optimal replica management strategy is then fed back to the process of evaluating each candidate edge node using a comprehensive cost function and the process of synchronizing data replicas between the cloud and edge nodes using federated learning technology, thus achieving closed-loop control.

[0016] As an alternative implementation, the static costs include the unit storage cost in the cloud, the overhead of using solid state disks (SSDs) at local edge nodes, and the limited storage capacity of terminal devices.

[0017] The dynamic costs include the central processing unit (CPU) utilization, memory and bandwidth utilization of edge nodes, and hotspot effects caused by data access frequency.

[0018] The business constraints include service level agreements and data compliance requirements for real-time machine learning tasks.

[0019] As an alternative implementation method, the comprehensive cost function is:

[0020] ;

[0021] in, Represents a data block. Indicates candidate edge nodes; Indicates storage cost, Indicates transmission cost, Indicates the cost of delay. This indicates penalties for violating service level agreements or compliance requirements. is the weighting coefficient, and the final optimal replica position is the edge node position that minimizes the comprehensive cost function.

[0022] As an alternative implementation, the following steps are also included: constructing a distributed hash ring, mapping the cloud, edge nodes and terminal devices to a virtual space on the hash ring, generating a unique identifier based on the content and timestamp of the data block, mapping the unique identifier to a location using the hash ring distance function, and constructing a replica location index.

[0023] As a further defined implementation method, data blocks The unique identifier is defined as:

[0024] ;

[0025] in, Represents the content characteristics of a data block (e.g., file digest, vector signature, or data fingerprint). For the timestamps generated or updated in the data, This represents the task identifier associated with the data block. This represents a splicing operation. This represents a large number hash function used to generate a unique fingerprint of a data block.

[0026] data blocks The mapping rule to a node on the distributed hash ring is as follows:

[0027] ;

[0028] in Represents the set of all candidate nodes. Indicates candidate edge nodes, A hash function is used to map physical nodes to virtual coordinates on a hash ring. Represents the hash ring distance function. Representing data and nodes The hash distance between them.

[0029] As an alternative implementation, the process of selecting several neighboring nodes as backup locations includes: introducing a redundant replica strategy and selecting 1 to N neighboring nodes as backup locations.

[0030] ;

[0031] ;

[0032] in, Represents a data block. For data blocks The set of backup locations, This indicates the edge node corresponding to the backup location. For data blocks The replica is stored on the node The associated total cost, where k is the number of neighboring nodes selected. To ensure that the cost value is acceptable, if a hash collision occurs, a secondary verification is performed by attaching the task ID and timestamp to ensure that the mapping relationship between the replica and the data is globally unique.

[0033] As an alternative implementation, the preset triggering conditions are that the load increment of the edge node exceeds a set value, the degree of change in the data access mode exceeds a set range, or the network topology fluctuates beyond a set range.

[0034] As an alternative implementation method, the process of migrating the corresponding copy includes: the migration strategy is: ;

[0035] in, Indicates the cost of migration. For migration cost weighting coefficients, For the comprehensive cost function, Represents a data block. These are the edge nodes before the migration. This indicates the edge node to which the migration has taken place.

[0036] As an alternative implementation, each edge node uses local data for model training and uploads the updated model parameters to the cloud for aggregation. When uploading the updated model parameters, the edge node first encrypts and signs them. After receiving the model parameters, the cloud performs hash verification and signature verification to ensure the integrity and correctness of the parameters.

[0037] As an alternative implementation method, the process of using the Byzantine fault tolerance mechanism to detect data copies and remove edge nodes with abnormal information synchronization includes: marking edge nodes whose transmitted data and expected model parameters have a deviation greater than a set range as abnormal nodes, and voting on them. If a set number of nodes believe that a certain model parameter is abnormal, then the model parameter is marked as untrustworthy.

[0038] Calculate the cosine similarity or L2 norm between the uploaded model parameters and the global model parameters in the cloud. If the cosine similarity or L2 norm exceeds the set range, the model parameter is marked as untrustworthy.

[0039] Edge nodes that upload model parameters marked as untrusted are considered edge nodes with information synchronization anomalies and are removed from subsequent aggregation and training processes. A reselection mechanism is then initiated to select edge nodes with reputation values ​​higher than a set value that have not been marked as anomalies to replace the removed edge nodes.

[0040] As an alternative implementation, the process of abstracting the data replica scheduling problem in a dynamic environment into a Markov decision process includes: utilizing quintuples Represent a Markov decision process, where the state space is... Used to characterize the operational state of the cloud edge environment at the target time, motion space For the set of executable replica management strategies, the reward function Taking into account resource utilization, load balancing, access latency, and replica availability, the state transition probability... Discount factor is used to represent the probability of transitioning from one state to the next. Used to balance short-term performance and long-term stability .

[0041] As a further limitation, the runtime state space at the target time... for:

[0042] ;

[0043] in, This shows the load status of each edge node. For the remaining storage capacity, For bandwidth and transmission delay, For the current replica distribution, For request queue length and latency characteristics;

[0044] Action space include:

[0045] A1 replica location adjustment: migrate hot data from overloaded edge nodes to nearby idle nodes;

[0046] A2 synchronization frequency optimization: dynamically adjust the data replica synchronization cycle;

[0047] A3 Redundancy Control: Adjusts the redundancy ratio of erasure codes for key model parameters;

[0048] A4 Copy Creation / Deletion: The proportion of pre-created standby copies or the proportion of deleted redundant copies;

[0049] A5's current management strategy remains unchanged.

[0050] As an alternative implementation, the optimization process includes using a deep deterministic policy gradient algorithm, which comprises a policy network and a value network, wherein the input to the policy network is the current state. Output continuous motion Control parameters are generated through a deterministic strategy; the input to the value network is state-action pairs. Used to assess long-term value It guides the optimization direction of the strategy network, updates the network parameters through the gradient descent algorithm, and guides the optimization of value estimation by minimizing the temporal difference error loss function.

[0051] A collaborative data replication control system for real-time machine learning in the cloud, comprising:

[0052] The multi-dimensional cost awareness module is configured to evaluate each candidate edge node using a comprehensive cost function for data replicas of data blocks. With the goal of minimizing the comprehensive cost function, it determines the optimal replica location and selects several neighboring nodes as backup locations. The comprehensive cost function is a weighted fusion of static cost, dynamic cost, and business constraints.

[0053] The intelligent real-time placement module is configured to acquire the real-time parameters of each edge node. When the real-time parameters of the edge node meet the preset trigger conditions, the data replicas are redistributed, the comprehensive cost function is re-evaluated, the optimal replica position is determined, and the corresponding replicas are migrated.

[0054] The cloud-edge replica security collaboration module is configured to use federated learning technology to synchronize data replicas between the cloud and edge nodes. The cloud model is shared on each edge node, and the cloud periodically broadcasts the latest parameters of the global model to each edge node. Each edge node uses local data to train the model and uploads the updated model parameters to the cloud for aggregation. If the data uploaded by an edge node deviates from the expected model parameters by more than a set range, the Byzantine fault tolerance mechanism is used to detect data replicas and remove edge nodes with abnormal information synchronization.

[0055] The data replica load balancing module is configured to abstract the data replica scheduling problem in a dynamic environment into a Markov decision process. It comprehensively considers resource utilization, load balancing, access latency, and replica availability to perform optimization and solve the problem. Based on the optimization results, it determines the optimal replica management strategy and feeds the optimal replica management strategy back to the multi-dimensional cost awareness module and the cloud-edge replica security collaboration module.

[0056] As an alternative implementation, a distributed hash ring and an unambiguous location module are also included, which are configured to construct a distributed hash ring, map the cloud, edge nodes and terminal devices to the virtual space on the hash ring, generate a unique identifier based on the content and timestamp of the data block, and use the hash ring distance function to map the unique identifier to the location and construct a replica location index.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0058] This invention enables cost-aware copy placement, secure and reliable collaboration in cloud-edge-device environments, and dynamic closed-loop control, thereby reducing storage and transmission overhead while ensuring low latency and high reliability, thus supporting real-time data processing and intelligent applications in industrial internet and edge intelligence scenarios.

[0059] The data replica placement of this invention is based on a comprehensive multi-objective evaluation of static cost, dynamic cost, and business constraints, resulting in accurate values. Furthermore, it dynamically adjusts the optimal placement of replicas under different node resource states and access modes. For hot data replicas, it supports rapid replica migration and reduction, ensuring efficient response. Both data replica placement and migration are based on dynamic rules, enabling dynamic responses to real-time changes in industrial tasks and timely optimization of replica distribution based on the status of edge nodes, thereby improving data processing efficiency.

[0060] This invention constructs a replica location index based on consistent hashing to avoid replica drift. At the same time, it combines an unambiguous replica definition to ensure consistent numbering and traceability of replicas in different cloud regions and edge nodes.

[0061] This invention utilizes federated learning technology to share cloud models on edge nodes and uses node data for local model training to ensure that sensitive data does not leave the domain. The cloud model asynchronously aggregates node model parameters for global model updates, and the edge stores and transmits data copies containing model parameters and non-sensitive data to ensure data exchange between the cloud and the edge.

[0062] This invention utilizes the Byzantine fault tolerance mechanism to ensure the reliability of edge nodes in confirming the status of data replicas in a distributed environment. Especially in the presence of malicious / abnormal nodes, it can effectively prevent forged synchronization messages and ensure the consistency and correctness of data replicas.

[0063] This invention abstracts the intelligent load balancing problem in dynamic environments into a Markov decision process, employs a deep deterministic policy gradient algorithm to continuously update the policy network parameters, achieves optimized solutions, and learns online and provides real-time feedback based on the optimization results. Through closed-loop control, it continuously accumulates environmental interaction data, optimizes the policy network, and achieves continuous improvement in system performance.

[0064] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0065] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0066] Figure 1 This is a flowchart of a data replica control system for real-time machine learning in the cloud, as proposed in one embodiment of the present invention.

[0067] Figure 2 This is a flowchart of a cost-aware and intelligent copy placement strategy proposed in one embodiment of the present invention;

[0068] Figure 3 This is a flowchart illustrating reliable collaboration between cloud-edge replicas according to one embodiment of the present invention.

[0069] Figure 4 This is a flowchart of a real-time feedback closed-loop control system driven by online reinforcement learning, proposed in one embodiment of the present invention. Detailed Implementation

[0070] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0071] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0072] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0073] Where there is no conflict, the embodiments and features described in this application may be combined with each other.

[0074] Example 1

[0075] A collaborative data replication control method for real-time machine learning in the cloud, such as Figure 1 As shown, it includes the following steps:

[0076] Efficient data replica placement: For data replicas of data blocks, a comprehensive cost function is used to evaluate each candidate edge node. The optimal replica location is determined with the goal of minimizing the comprehensive cost function, and several neighboring nodes are selected as backup locations. The comprehensive cost function is a weighted fusion of static cost, dynamic cost, and business constraints.

[0077] The system acquires real-time parameters of each edge node. When the real-time parameters of the edge node meet the preset triggering conditions, it redistributes data replicas, re-evaluates the comprehensive cost function, determines the optimal replica location, and migrates the corresponding replicas.

[0078] Cloud-edge replica security collaboration: Federated learning technology is used to synchronize data replicas between the cloud and edge nodes. The cloud model is shared on each edge node, and the cloud periodically broadcasts the latest parameters of the global model to each edge node. Each edge node uses local data to train the model and uploads the updated model parameters to the cloud for aggregation. If the data uploaded by an edge node deviates from the expected model parameters by more than a set range, the Byzantine fault tolerance mechanism is used to detect data replicas and remove edge nodes with abnormal information synchronization.

[0079] Data replica load balancing stage: The data replica scheduling problem in a dynamic environment is abstracted into a Markov decision process. It comprehensively considers resource utilization, load balancing, access latency, and replica availability to optimize the solution. Based on the optimization result, the optimal replica management strategy is determined. The optimal replica management strategy is fed back to the process of evaluating each candidate edge node using a comprehensive cost function and the process of synchronizing data replicas between the cloud and edge nodes using federated learning technology (i.e., the feedback optimization of efficient data replica placement stage and the cloud-edge replica security collaboration stage), to achieve closed-loop control.

[0080] The following is a detailed explanation of each step.

[0081] First is the multi-dimensional cost perception process, such as Figure 2 As shown, it uses a weighted multi-objective function to comprehensively score potential placement locations, ultimately determining the optimal replica distribution strategy. Specifically, static costs include unit storage costs in the cloud, the overhead of solid-state drives (SSDs) on local edge nodes (hereinafter referred to as nodes), and the limited storage capacity of terminal devices; dynamic costs cover the utilization of the node's central processing unit (CPU), memory, and bandwidth, as well as the hotspot effect caused by data access frequency; in addition, business constraints must be met, such as Service Level Agreements (SLAs) and data compliance requirements for real-time machine learning tasks. The comprehensive evaluation function is dynamically adjusted with weight parameters, enabling the system to flexibly balance latency and cost according to the task requirements of different stages such as training and inference. Its comprehensive cost function (i.e., cost-aware model) is defined as:

[0082] ;

[0083] in, Represents a data block. Indicates candidate nodes; Indicates storage cost, Indicates transmission cost, Indicates the cost of delay. This indicates penalties for violating SLAs or compliance constraints. Weight The system is dynamically adjusted according to the business stage to achieve an adaptive balance between different optimization goals.

[0084] This embodiment presents a dynamic adjustment process that improves efficiency when storage space at edge nodes is limited. This prompts the system to prioritize nodes with lower storage costs or more abundant storage resources; while during model training or inference phases, to improve computational efficiency and data locality, the storage cost can be appropriately reduced. .

[0085] When network congestion or bandwidth limitations occur, improve This reduces cross-node data transmission; if the system operates in a low-latency, high-bandwidth network environment, the corresponding reduction will be achieved. .

[0086] Weight Dynamically set according to task type: For real-time control or online inference tasks, assign higher weight (e.g., ...). Prioritize latency performance; for offline training or batch processing tasks, lower weights can be set (e.g., ...). ).

[0087] If a node is detected to have potential data leakage or trust risks, then increase... This is to strengthen the constraints on security and compliance.

[0088] All weights satisfy the normalization constraint:

[0089] ;

[0090] This ensures that the system achieves overall optimization in terms of storage costs, transmission overhead, latency performance, and security risks.

[0091] The final optimal replica location is:

[0092] ;

[0093] That is, the edge node with the minimum comprehensive cost function is selected as the optimal replica position.

[0094] In this embodiment, a consistent hashing extension mechanism is introduced during the replica mapping phase to construct a geographically aware distributed hash ring (Geo-DHT). Through this mechanism, cloud, edge nodes, and terminal devices are mapped to a virtual space on the hash ring. The placement of replicas is determined by latitude, longitude, and node roles, thereby achieving efficient replica allocation in cross-domain environments. (Data block) The unique identifier is defined as:

[0095] ;

[0096] in, Represents the content characteristics of a data block (e.g., file digest, vector signature, or data fingerprint). For the timestamps generated or updated in the data, This represents the task identifier associated with the data block. This represents a splicing operation. A large number hash function (such as SHA-256) is used to generate a unique fingerprint of a data block.

[0097] data blocks The mapping rule to a node on the distributed hash ring is as follows:

[0098] ;

[0099] in Represents the set of all candidate nodes. Indicates candidate edge nodes, A hash function is used to map physical nodes to virtual coordinates on a hash ring. Represents the hash ring distance function. Representing data and nodes The hash distance between them.

[0100] To avoid ambiguity in replicas, an unambiguity rule is proposed. Each data block generates a unique identifier based on its content and timestamp, which is then mapped to the nearest node on the ring via the SHA-256 hash function.

[0101] To further enhance the reliability and fault tolerance of the replicas, this embodiment also introduces a redundant replica strategy, selecting 1 to N neighboring nodes as backup locations, i.e.:

[0102] ;

[0103] ;

[0104] in, Represents a data block. For data blocks The set of backup locations, This indicates the edge node corresponding to the backup location. For data blocks The replica is stored on the node The associated total cost, where k is the number of neighboring nodes selected. To ensure that the cost value is acceptable, if a hash collision occurs, a secondary verification is performed by attaching the task ID and timestamp to ensure that the mapping relationship between the replica and the data is globally unique.

[0105] To adapt to dynamic changes in complex environments, this embodiment also proposes a real-time dynamic adjustment mechanism. When a sudden increase in node load, a drastic change in data access patterns, or a significant fluctuation in network topology is detected, a replica redistribution process is triggered, i.e., a replica adjustment request is generated. The triggering conditions can be formalized as follows:

[0106] in For node load, For access frequency, For link bandwidth, For node load limits, For time intervals, To limit the access frequency, if the conditions are met, the execution process includes: first, the monitoring module collects the real-time parameters of each node and recalculates the score; then, Geo-DHT quickly locates the optimal position of the replica; finally, only replicas with large differences are migrated, thereby ensuring performance while avoiding the additional costs caused by frequent migrations.

[0107] The migration strategy can be formalized as: ;

[0108] in, Indicates the cost of migration. For migration cost weighting coefficients, For the comprehensive cost function, Represents a data block. These are the edge nodes before the migration. This indicates the edge node to which the migration has taken place.

[0109] like Figure 3 As shown, the cloud periodically broadcasts the latest parameters of the global model (such as the weights of the neural network model, updated configurations, etc.) to each edge node. Each edge node uses local data for model training; this raw data is always stored on the edge node and is never uploaded to the cloud, thus ensuring data privacy. After local training, the edge node only sends the updated model parameters back to the cloud for aggregation and summarization.

[0110] To ensure the security and integrity of the federated learning process, edge nodes encrypt and sign the local model parameters before uploading them to prevent data tampering during transmission. Encryption ensures that even if the parameters are intercepted during transmission, unauthorized parties cannot read or modify them; the signature verifies the source's credibility, ensuring that the parameters indeed come from a legitimate edge node. Upon receiving the model parameters, the cloud verifies their integrity and correctness through hash verification and signature verification. Hash verification detects whether the uploaded data has been tampered with during transmission, while signature verification confirms that the data originates from the expected edge node, thus preventing malicious nodes from forging or tampering with model updates.

[0111] Byzantine fault tolerance is introduced to ensure the consistency of data replicas and eliminate the influence of untrusted or malicious nodes. If the data uploaded by some nodes deviates significantly from the expected model results, the system will mark the node as an abnormal node and perform the following checks:

[0112] Multi-node voting mechanism: The cloud verifies model consistency through a voting mechanism among multiple nodes. If enough nodes believe that a certain model parameter is abnormal, that parameter will be marked as untrustworthy.

[0113] Model consistency detection: By calculating the cosine similarity or L2 norm between the uploaded model and the global model in the cloud, the cloud can quickly detect abnormal updates.

[0114] If malicious behavior is detected in a node through multi-party verification, that node will be removed and withdrawn from subsequent aggregation and training processes. Simultaneously, the system will initiate a reselection mechanism, choosing a node with higher credibility that has not been marked as abnormal to replace the malicious node, thereby ensuring the integrity and consistency of model aggregation.

[0115] Ultimately, all validated model updates are weighted and aggregated based on node reputation scores and data consistency to ensure that the global model reflects the true situation of most nodes.

[0116] ;

[0117] in, For the set of nodes that have passed verification, For nodes Reputation score, The model parameters uploaded to this node. This represents the parameters after global model aggregation.

[0118] This embodiment also uses online reinforcement learning to drive real-time feedback closed-loop control. The data copy scheduling problem in online reinforcement learning can be abstracted as a Markov Decision Process (MDP), using quintuples. To express.

[0119] state space Used to characterize the operational state of an industrial cloud-edge environment at a given moment:

[0120] ;

[0121] in, This shows the load status of each node (CPU, memory, input / output, etc.). For the remaining storage capacity, For bandwidth and transmission delay, For the current replica distribution, This refers to the request queue length and latency characteristics.

[0122] Action space A set of executable replica management policies, including:

[0123] Replica location adjustment: migrate hot data from overloaded edge nodes to nearby idle nodes;

[0124] Synchronization frequency optimization: Dynamically adjust the data replica synchronization cycle;

[0125] Redundancy control: Adjust the redundancy ratio of erasure codes for key model parameters;

[0126] Replica creation / deletion: The proportion of pre-created standby replicas or the proportion of deleted redundant replicas;

[0127] It remains unchanged.

[0128] reward function Taking into account resource utilization, load balancing, access latency, and replica availability:

[0129] ;

[0130] in Indicates system resource utilization; Indicates the degree of load imbalance; Indicates average access latency; Replica availability and data reliability; This is a weighting coefficient, adjusted according to the importance of the system objective.

[0131] If the system prioritizes resource utilization, it can be appropriately increased. The value of should be chosen to encourage task migration and full utilization of resources; if the system has high requirements for load balancing, then should be increased. The weighting of nodes should be adjusted to reduce the impact of uneven node load and promote even task distribution and efficient processing; if the system prioritizes low latency performance, then the weighting should be increased. The weighting should be prioritized to ensure response time and real-time requirements. If the system's primary objective is data reliability, then the weighting should be increased. This enhances replica consistency and fault tolerance. To avoid any single indicator having an excessively large or small impact on overall decision-making, all weight coefficients should satisfy normalization constraints:

[0132] ;

[0133] This constraint ensures that the system achieves stable trade-offs and adaptive control among different optimization objectives. For example, in a system where the primary objective is to balance resource utilization and data reliability, and where there is a certain tolerance for load imbalance and access latency, the following weight configuration can be set: This configuration maintains the system's high efficiency while ensuring stability and reliability, enabling the system to achieve an optimal balance between performance, cost, and security in complex cloud-edge-device collaborative environments.

[0134] State transition probability It depicts the evolution of the system from one state to the next, with the transition determined by both the result of the action and the dynamic changes in the environment, i.e., given the current state. and the actions taken The system enters a new state at the next moment. The probability of:

[0135] ;

[0136] Discount factor Used to balance short-term performance and long-term stability ,when When the latency is close to 0, the system focuses more on short-term rewards, such as quickly reducing current latency; conversely, the system focuses more on long-term performance, such as the stability of replica distribution and maximizing resource utilization.

[0137] To achieve efficient optimization in a continuous action space, the Deep Deterministic Policy Gradient (DDPG) algorithm is chosen. For example... Figure 4 As shown, DDPG adopts an Actor-Critic architecture, where the Actor network (policy network) takes the current state as input. Output continuous motion Precise control parameters are directly generated through deterministic strategies; Critic networks (value networks) input state-action pairs. Assess long-term value This guides the optimization direction of the Actor network; the target network includes the target Actor network and the target Critic network, and the training process is stabilized through soft updates to avoid oscillations in Q-value estimation.

[0138] The core of online learning optimization is to continuously optimize the policy network using interactive data through an incremental learning mechanism, enabling the system to dynamically adapt to real-time changes in the cloud-edge industrial environment. Each time the agent (decision engine) completes a replica management action, the system records its current state. Actions performed Instant rewards received and the next state and this quadruple ( Stored in an experience replay buffer. This buffer stores a wealth of experience from historical interactions, including both high-value samples that significantly reduce latency and ordinary or low-value samples that do not show obvious effects.

[0139] The decision engine periodically samples a batch of samples randomly from the buffer and updates the Actor-Critic network parameters of DDPG using a gradient descent algorithm. The Critic network is responsible for evaluating state-action pairs. long-term value The optimization of value estimation is guided by minimizing the temporal difference error loss function:

[0140] ;

[0141] in , γ As a discount factor, Q The Critic network is the target network; the Actor network, based on feedback from the Critic network, maximizes action value. The gradient ascent update strategy generates better replica management actions.

[0142] As interactive data accumulates, the policy network gradually learns the optimal decision-making patterns for different industrial scenarios (such as high-load production during day shifts and low-activity maintenance during night shifts). This continuous optimization mechanism based on incremental learning enables the system to gradually improve its decision-making capabilities through a cycle of trial and error, feedback, and improvement, just like human experts, adapting to the long-term dynamic changes in the industrial environment.

[0143] Example 2

[0144] A collaborative data replication control system for real-time machine learning in the cloud, comprising:

[0145] The multi-dimensional cost awareness module is configured to evaluate each candidate edge node using a comprehensive cost function for data replicas of data blocks. With the goal of minimizing the comprehensive cost function, it determines the optimal replica location and selects several neighboring nodes as backup locations. The comprehensive cost function is a weighted fusion of static cost, dynamic cost, and business constraints.

[0146] The intelligent real-time placement module is configured to acquire the real-time parameters of each edge node. When the real-time parameters of the edge node meet the preset trigger conditions, the data replicas are redistributed, the comprehensive cost function is re-evaluated, the optimal replica position is determined, and the corresponding replicas are migrated.

[0147] The cloud-edge replica security collaboration module is configured to use federated learning technology to synchronize data replicas between the cloud and edge nodes. The cloud model is shared on each edge node, and the cloud periodically broadcasts the latest parameters of the global model to each edge node. Each edge node uses local data to train the model and uploads the updated model parameters to the cloud for aggregation. If the data uploaded by an edge node deviates from the expected model parameters by more than a set range, the Byzantine fault tolerance mechanism is used to detect data replicas and remove edge nodes with abnormal information synchronization.

[0148] The data replica load balancing module is configured to abstract the data replica scheduling problem in a dynamic environment into a Markov decision process. It comprehensively considers resource utilization, load balancing, access latency, and replica availability to perform optimization and solve the problem. Based on the optimization results, it determines the optimal replica management strategy and feeds the optimal replica management strategy back to the multi-dimensional cost awareness module and the cloud-edge replica security collaboration module.

[0149] In this embodiment, a distributed hash ring and unambiguous positioning module are also included. The module is configured to construct a distributed hash ring, map the cloud, edge nodes and terminal devices to the virtual space on the hash ring, generate a unique identifier based on the content and timestamp of the data block, and use the hash ring distance function to map the unique identifier to the location and construct a replica positioning index.

[0150] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of one or more computer-usable storage media (including, but not limited to, disk storage, etc.) containing computer-usable program code. CD - ROM It takes the form of a computer program product implemented on (such as optical memory, etc.).

[0151] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0152] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0154] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art without creative effort within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A collaborative data replica control method for real-time machine learning on cloud, characterized in that, Comprise the following steps: For the data copy of the data block, each candidate edge node is evaluated by using a comprehensive cost function, and the optimal copy position is determined by minimizing the comprehensive cost function, and a plurality of adjacent nodes are selected as backup positions, and the comprehensive cost function is weighted and combined with static cost, dynamic cost and service constraint; The static cost includes the unit storage cost of the cloud, the solid state disk usage overhead of the local edge node and the limited storage capacity of the terminal device; The dynamic cost includes the central processing unit, memory and bandwidth utilization of the edge node, and the hotspot effect caused by data access frequency; The service constraint includes the service level agreement and data compliance requirement of real-time machine learning task; Real-time parameters of each edge node are obtained, and when the real-time parameters of the edge node meet the preset trigger condition, data copy redistribution is performed, the comprehensive cost function is re-evaluated, the optimal copy position is determined, and the corresponding copy is migrated; The synchronization of data copies between the cloud and the edge nodes is performed by using federated learning technology, the cloud model is shared on each edge node, the cloud regularly broadcasts the latest parameters of the global model to each edge node, each edge node trains the model using local data, and uploads the updated model parameters to the cloud for aggregation and aggregation; if there is a deviation greater than a set range between the data uploaded by the edge node and the expected model parameters, the Byzantine fault tolerance mechanism is used for data copy detection, and the edge node with abnormal information synchronization is removed; The data copy scheduling problem in the dynamic environment is abstracted as a Markov decision process, and the resource utilization, load balancing degree, access delay and copy availability are comprehensively considered for optimization and solution, and the optimal copy management strategy is determined according to the result of optimization and solution, and the optimal copy management strategy is fed back to the process of evaluating each candidate edge node by using the comprehensive cost function and the process of synchronizing the data copies between the cloud and the edge nodes by using the federated learning technology, to realize closed-loop control.

2. The method of claim 1, wherein the method is characterized by, Further comprise the following steps: A distributed hash ring is constructed, the cloud, edge nodes and terminal devices are mapped to a virtual space on the hash ring, a unique identifier is generated according to the content and timestamp of the data block, the unique identifier is mapped to a position by using a hash ring distance function, and a copy positioning index is constructed; Data block The unique identification of the data block is defined as: ； wherein, represents a content characteristic of the data block, is a timestamp of the data generation or update, represents a task identification associated with the data block, represents a concatenation operation, represents a large number hash function for generating a unique fingerprint of the data block; data blocks The mapping rule to a node on the distributed hash ring is as follows: ； wherein denotes a set of all candidate nodes, denotes a candidate edge node, is a hash function identifying nodes, mapping physical nodes to virtual coordinates of a hash ring, denotes a hash ring distance function, denotes data and a hash distance between nodes .

3. The method of claim 1, wherein the method is characterized by, The process of selecting a plurality of adjacent nodes as backup positions includes: introducing a redundant copy strategy, and selecting 1-N adjacent nodes as backup positions: ； ； wherein, represents a data block, is a backup location set of the data block , represents an edge node corresponding to the backup location, is a copy of the data block stored in the node associated with the total cost, k is the number of selected adjacent nodes, is an allowed cost value, if a hash collision occurs, a secondary check is performed by adding a task ID and a timestamp to ensure that the mapping relationship between the copy and the data is globally unique.

4. The method of claim 1, wherein the method is characterized by, The preset trigger condition is that the load increment of the edge node exceeds a set value, the data access mode variation degree exceeds a set range, or the network topology fluctuates more than a set range; The process of migrating the respective copy includes migrating the policy for: ; wherein, represents a migration cost, is a migration cost weight coefficient, is a comprehensive cost function, represents a data block, is an edge node before migration, represents an edge node to which migration is performed.

5. The method of claim 1, wherein the method is characterized by, The process of detecting the data copy by using the Byzantine fault tolerance mechanism and removing the edge node with abnormal information synchronization includes: marking the edge node with a deviation greater than a set range between the transmitted data and the expected model parameters as an abnormal node, and voting, if a set number of nodes think that a certain model parameter is abnormal, the model parameter is marked as untrusted; The cosine similarity or L2 norm between the uploaded model parameters and the global model parameters in the cloud is calculated, and if the cosine similarity or L2 norm exceeds a set range, the model parameters are marked as untrustworthy; The edge node that uploads the model parameters marked as untrustworthy is regarded as an edge node with information synchronization exception, is removed from subsequent aggregation and training processes, and a reselection mechanism is started to select an edge node with a higher reputation than a set value and without being marked as abnormal to replace the removed edge node.

6. The method of claim 1, wherein the method is characterized by, The process of abstracting the data replica scheduling problem in dynamic environment as a Markov decision process includes: using a five-tuple represents a Markov decision process, wherein the state space is used to describe the running state of the cloud edge environment at the target time, the action space is a set of executable replica management strategies, the reward function comprehensively considers resource utilization, load balancing, access delay and replica availability, and the state transition probability is used to represent the probability of a state to the next state, and the discount factor is used to balance the short-term performance and long-term stability, .

7. The method of claim 6, wherein the method is characterized by, operating state space at the target time point is: ； wherein, is a load situation of each edge node, is a remaining storage capacity, is a bandwidth and transmission delay, is a current replica distribution, is a request queue length and delay characteristic; Action space comprises: A1: Adjusting the replica location: migrating hot data from an overloaded edge node to a nearby idle node; A2: Optimizing the synchronization frequency: dynamically adjusting the data replica synchronization period; A3: Controlling the redundancy: adjusting the redundancy ratio of erasure coding for critical model parameters; A4: Creating / deleting replicas: pre-creating a proportion of backup replicas or deleting a proportion of redundant replicas; A5: The current management strategy remains unchanged.

8. The method of claim 1, wherein the method is characterized by, The process of optimization solving includes optimization solving by using a deep deterministic policy gradient algorithm, the deep deterministic policy gradient algorithm includes a policy network and a value network, wherein the input of the policy network is a current state , the output is a continuous action , a control parameter is generated through a deterministic policy; the input of the value network is a state-action pair , used for evaluating long-term value , guiding the optimization direction of the policy network, each network parameter is updated through a gradient descent algorithm, and the optimization of value estimation is guided by minimizing a time difference error loss function.

9. A collaborative data replica control system for real-time machine learning on the cloud, comprising: A multi-dimensional cost perception module configured to evaluate each candidate edge node for the data replica of a data block using a comprehensive cost function, determine the optimal replica location with the minimum comprehensive cost function, and select a number of nearby nodes as backup locations, wherein the comprehensive cost function integrates the static cost, dynamic cost, and business constraints; The static cost includes the unit storage fee of the cloud, the solid state disk usage overhead of the local edge node, and the storage capacity limited by the terminal device; The dynamic cost includes the central processing unit, memory, and bandwidth utilization rate of the edge node, as well as the hotspot effect caused by data access frequency; The business constraints include the service level agreement and data compliance requirements of the real-time machine learning task; An intelligent real-time placement module configured to obtain real-time parameters of each edge node, perform data replica redistribution when the real-time parameters of the edge node meet a preset trigger condition, reevaluate the comprehensive cost function, determine the optimal replica location, and migrate the corresponding replica; A cloud-edge replica security collaboration module configured to synchronize the data replicas between the cloud and the edge nodes using federated learning technology, share the cloud model on each edge node, periodically broadcast the latest parameters of the global model to each edge node, and use the local data for model training, upload the updated model parameters to the cloud for aggregation and training; if there is a deviation greater than a set range between the uploaded data and the expected model parameters of the edge node, use the Byzantine fault tolerance mechanism to detect the data replica and remove the edge node with information synchronization exception; A data replica load balancing module configured to abstract the data replica scheduling problem in a dynamic environment as a Markov decision process, comprehensively consider resource utilization, load balancing, access delay, and replica availability, and optimize the solution, determine the optimal replica management strategy according to the optimized solution, and feed the optimal replica management strategy back to the multi-dimensional cost perception module and the cloud-edge replica security collaboration module.

Images