Cloud-based data synchronization method and system

Abstract

The invention relates to the technical field of data processing, in particular to a cloud-based data synchronization method and system, and the method comprises the following steps: collecting a node set in a cloud environment and the conflict probability of each node in the data synchronization process, and adjusting the data synchronization conflict detection frequency to obtain conflict occurrence probability data; according to the method, the data synchronization conflict detection frequency of the nodes is dynamically adjusted, and the conflict occurrence probability is generated in combination with the update frequency of the nodes, the network state and other multi-dimensional data, so that the problem of applicability of fixed frequency detection to different nodes is avoided, and accurate adaptation to dynamic behaviors of different nodes is realized. By combining the network condition of the node, the synchronization time delay record and the size of the real-time task data, the influence of the conflict on the data synchronization process is evaluated, and the synchronization time delay change trend of the target path is predicted, so that the synchronization efficiency of path selection is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a cloud-based data synchronization method and system. Background Art

[0002] The field of data processing technology includes data acquisition, storage, transmission, analysis and other aspects, mainly involving the operation and management of various types of data. With the rapid development of information technology, the amount of data generated and stored continues to increase, especially in the fields of cloud computing, big data, and the Internet of Things, and data synchronization issues are becoming increasingly prominent. The core content of data synchronization technology includes ensuring the consistency, timeliness and reliability of data between different systems or devices, involving data transmission and updates on multiple platforms and multiple terminals.

[0003] The data synchronization method refers to a technical means to solve the problem of data synchronization between multiple devices or systems. In order to solve the technical problem of data synchronization between multiple devices in a distributed environment, the method uses specific algorithms and protocols to ensure that the data of each node is synchronized in a timely and accurate manner when data is exchanged between multiple nodes.

[0004] In the prior art, the data synchronization process usually uses fixed-frequency conflict detection to handle conflicts between nodes, which is difficult to adjust according to the dynamic behavior of the nodes, resulting in insufficient timely conflict handling for high-frequency changing nodes and waste of detection resources for low-frequency changing nodes. In addition, in a multi-node distributed environment, the selection of synchronization paths in the prior art often lacks a comprehensive evaluation of path delay, bandwidth and node load conditions, which can easily lead to irrational path selection, thereby increasing the synchronization failure rate or delaying transmission efficiency. In the task allocation and storage links, the prior art usually adopts a static allocation strategy, which fails to effectively associate the real-time load status of the node with the storage hierarchy, which can easily lead to excessive use of high-load node resources or uneven distribution of storage resources, resulting in reduced synchronization performance. For example, when the bandwidth utilization of a node is higher than 70%, the prior art fails to adjust the allocation ratio in a timely manner, which may cause task delays and even the risk of data loss. Summary of the invention

[0005] In order to solve the technical problems existing in the prior art, the embodiments of the present invention provide a cloud-based data synchronization method and system. The technical solution is as follows: The cloud-based data synchronization method includes the following steps: S1: Collect the node set in the cloud environment, the probability of conflict in each node during data synchronization, and adjust the frequency of data synchronization conflict detection to obtain conflict probability data; S2: combining the network status of each node in the conflict probability data, evaluating the impact of the conflict during data synchronization, predicting the future synchronization delay of the target synchronization path according to the evaluation result, and generating a delay trend analysis result; S3: According to the delay trend analysis result, select a corresponding synchronization path for data transmission, and evaluate the current node load status in real time during the data transmission process to generate path selection and load evaluation data; S4: Based on the path selection and load evaluation data, monitor the load and bandwidth status of all nodes, dynamically adjust the allocation of data transmission tasks according to the load and bandwidth status of the nodes, and generate scheduling priority and resource allocation data; S5: Based on the scheduling priority and resource allocation data, the data synchronization tasks of each priority level are allocated to storage nodes of corresponding levels in the cloud platform to generate cloud storage level synchronization data.

[0006] The present invention has improvements in that the conflict occurrence probability data includes node update frequency, network delay, and node load status; the delay trend analysis result includes the delay value of the target synchronization path, the influence coefficient of the data transmission size on the path, and the weighted value of the node processing capability to the delay; the path selection and load evaluation data includes the path real-time synchronization quality score, the node load status score, and the bandwidth utilization of the transmission path; the scheduling priority and resource allocation data includes the task scheduling priority weight, the node bandwidth allocation ratio, and the node resource utilization after task allocation; the cloud storage level synchronization data includes cache task sharding data, distributed storage task processing data, and storage task archiving data.

[0007] The present invention is improved in that the specific steps of collecting the node set in the cloud environment, the probability of conflict of each node during data synchronization, and adjusting the frequency of data synchronization conflict detection to obtain the conflict probability data are as follows: S101: Based on the node set in the cloud environment, statistically analyze the update frequency of the nodes, and record the update timestamp of each node in real time to generate node update frequency data; S102: Based on the node update frequency data and in combination with the network status of the node, dynamically evaluate the change frequency of the node, and associate the evaluation result with the node information to generate node change frequency evaluation data; S103: Based on the node change frequency evaluation data and in combination with the conflict probability of the node during data synchronization, the conflict detection frequency of the node is adjusted, and the adjusted conflict detection frequency is recorded to generate conflict occurrence probability data.

[0008] The present invention has the following improvements: based on the node change frequency evaluation data, the formula is adopted: ; Compute the probability of conflict between nodes during data synchronization ; in, Indicates the update change rate of the node, Indicates the frequency of node conflict detection. It represents the sensitivity coefficient of the probability of conflict occurrence to the change of detection frequency. represents the reference detection frequency of the node, is the base of natural logarithms.

[0009] The present invention is improved in that the network status of each node in the conflict probability data is combined to evaluate the impact of the conflict during data synchronization, and the future synchronization delay of the target synchronization path is predicted according to the evaluation result. The specific steps of generating the delay trend analysis result are as follows: S201: Based on the conflict probability data, combined with the network status of each node, the synchronization delay record and the data size of the real-time synchronization task, the conflict impact of the current synchronization is evaluated to generate conflict impact evaluation data; S202: Based on the conflict impact assessment data and in combination with the processing capability information of the nodes, the synchronization behavior of the nodes in the target path is predicted to generate a target path synchronization delay trend analysis result; S203: Based on the target path synchronization delay trend analysis result, combined with the historical delay records of each node and the current network status, the delay change trend of the target path is summarized and analyzed to generate a delay trend analysis result.

[0010] The present invention is improved in that, according to the delay trend analysis result, the corresponding synchronization path is selected for data transmission, and the current node load state is evaluated in real time during the data transmission process. The specific steps of generating path selection and load evaluation data are as follows: S301: Based on the delay trend analysis result, the current transmission quality of each synchronization path is measured and evaluated in real time, the quality parameters of the path are recorded, and the path transmission quality information is generated; S302: Based on the path transmission quality information and the network status, a synchronization quality score of each path is calculated by weighted combination, and the scoring result is associated with the path information to generate a path synchronization quality score; S303: Based on the path synchronization quality score, select the path with the highest synchronization score for data transmission, and monitor the load status of the node during the transmission process in real time, record the load change of the node and the transmission process, and generate path selection and load evaluation data.

[0011] The present invention is improved in that the synchronization quality score of each path is calculated by using the formula: ; Get the synchronization quality score of the target path ; in, They represent the weight parameters of delay, packet loss rate and bandwidth utilization in path synchronization quality, represents the average transmission delay of the path, Indicates the proportion of packet loss in data transmission along the path Indicates the average bandwidth usage of the path, are the maximum and minimum values ​​of the delay, are the maximum and minimum values ​​of the packet loss rate, These are the maximum and minimum values ​​of bandwidth utilization.

[0012] The present invention is improved in that, based on the path selection and load evaluation data, the load and bandwidth conditions of all nodes are monitored, the allocation of data transmission tasks is dynamically adjusted according to the load and bandwidth conditions of the nodes, and the specific steps of generating scheduling priority and resource allocation data are as follows: S401: Based on the path selection and load evaluation data, monitor and collect statistics on the real-time load and bandwidth status of all nodes to generate node load and bandwidth status; S402: Based on the node load and bandwidth status, dynamically allocate and adjust the data transmission task, refine and associate the node load and bandwidth change trend, and generate a dynamic task allocation result; S403: Based on the dynamic task allocation result, the scheduling priority of each data transmission task is reallocated, and the scheduling priority and resource allocation data are generated in combination with the current load and bandwidth resource allocation situation.

[0013] The present invention is improved in that, based on the scheduling priority and resource allocation data, the data synchronization tasks of each priority are allocated to the storage nodes of the corresponding level in the cloud platform, and the specific steps of generating cloud storage level synchronization data are as follows: S501: Based on the scheduling priority and resource allocation data, the data synchronization tasks of each priority are associated and matched with the storage nodes of the corresponding level in the cloud platform, the corresponding relationship between the priority and the storage node is analyzed, and the mapping result between the priority and the storage node is generated according to the analysis result; S502: Based on the mapping result between the priority and the storage node, data sharding is performed on the data synchronization content of the task according to the priority order, and the storage location of the sharded data is allocated according to the storage node capacity to generate priority sharding and storage location information; S503: Based on the priority sharding and storage location information, the sharding data is synchronously transmitted to the corresponding storage node, and the hierarchical storage structure information after the transmission is completed is recorded to generate cloud storage hierarchical synchronization data.

[0014] A cloud-based data synchronization system, the system comprising: The conflict detection and adjustment module collects the node set in the cloud environment, calculates the probability of conflict for each node during data synchronization, adjusts the frequency of data synchronization conflict detection, and obtains conflict probability data; The delay prediction module combines the network status of each node in the conflict probability data, evaluates the impact of the conflict during data synchronization, predicts the future synchronization delay of the target synchronization path, and generates a delay trend analysis result; The path selection and load evaluation module selects the corresponding synchronization path for data transmission according to the delay trend analysis result, evaluates the current node load status in real time during the data transmission process, and generates path selection and load evaluation data; The resource scheduling optimization module monitors the load and bandwidth status of all nodes based on the path selection and load evaluation data, dynamically adjusts the allocation of data transmission tasks according to the load and bandwidth status of the nodes, and generates scheduling priority and resource allocation data; The cloud storage hierarchical synchronization module allocates data synchronization tasks of each priority level to storage nodes of corresponding levels in the cloud platform based on the scheduling priority and resource allocation data, and generates cloud storage hierarchical synchronization data.

[0015] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: By dynamically adjusting the data synchronization conflict detection frequency of the node, and combining the node update frequency, network status and other multi-dimensional data to generate the probability of conflict, the applicability of fixed frequency detection to different nodes is avoided, and accurate adaptation to the dynamic behavior of different nodes is achieved. By combining the node's network status, synchronization delay record and real-time task data size, the impact of conflicts on the data synchronization process is evaluated, and the synchronization delay change trend of the target path is predicted, effectively improving the synchronization efficiency of path selection. During the data transmission process, by real-time evaluation of the node load status and dynamic adjustment of data allocation tasks, load balancing is ensured to avoid performance bottlenecks caused by overload operation of individual nodes. In the priority and resource allocation link, through sharding processing and hierarchical matching of storage nodes, storage utilization and data transmission efficiency are effectively improved, resource waste is reduced, and the transmission timeliness of high-priority tasks is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 is a flow chart of the method of the present invention; Figure 2 This is a detailed flow chart of step S1 of the present invention; Figure 3 This is a detailed flow chart of step S2 of the present invention; Figure 4 This is a detailed flow chart of step S3 of the present invention; Figure 5 This is a detailed flow chart of step S4 of the present invention; Figure 6 This is a detailed flow chart of step S5 of the present invention; Figure 7 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0019] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0020] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0021] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0022] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0023] See also Figure 1 , an embodiment of the present invention provides a cloud-based data synchronization method, comprising the following steps: S1: Collect the node set in the cloud environment, the probability of conflict in each node during data synchronization, and adjust the frequency of data synchronization conflict detection to obtain conflict probability data; S2: Combine the network status of each node in the conflict probability data to evaluate the impact of the conflict during data synchronization, predict the future synchronization delay of the target synchronization path based on the evaluation results, and generate delay trend analysis results; S3: According to the delay trend analysis results, the corresponding synchronization path is selected for data transmission, and the current node load status is evaluated in real time during the data transmission process to generate path selection and load evaluation data; S4: Based on the path selection and load evaluation data, monitor the load and bandwidth status of all nodes, dynamically adjust the allocation of data transmission tasks according to the load and bandwidth status of the nodes, and generate scheduling priority and resource allocation data; S5: Based on the scheduling priority and resource allocation data, the data synchronization tasks of each priority level are assigned to the storage nodes of the corresponding level in the cloud platform to generate cloud storage level synchronization data; The conflict probability data include node update frequency, network delay, and node load status. The delay trend analysis results include the delay value of the target synchronization path, the influence coefficient of data transmission size on the path, and the weighted value of node processing capacity on the delay. The path selection and load assessment data include the path real-time synchronization quality score, node load status score, and bandwidth utilization of the transmission path. The scheduling priority and resource allocation data include task scheduling priority weight, node bandwidth allocation ratio, and node resource utilization after task allocation. The cloud storage level synchronization data includes cache task sharding data, distributed storage task processing data, and storage task archiving data.

[0024] See also Figure 2 , collect the node set in the cloud environment, the probability of conflict in each node during data synchronization, and adjust the frequency of data synchronization conflict detection. The specific steps to obtain the conflict probability data are as follows: S101: Based on the node set in the cloud environment, statistically analyze the update frequency of the nodes, and record the update timestamp of each node in real time to generate node update frequency data; By setting a fixed time window (for example, 1 hour or 1 day), all update events of each node within the time window are recorded. First, the unique identifier of each node and its corresponding update time data are extracted, and these data are sorted in chronological order. The update frequency distribution in the time series is analyzed through the sliding window technology, and the time period is divided into multiple intervals using the grouping statistical method. The number of update events in each interval is accumulated to form an update frequency value. At the same time, the timestamp of each update event is recorded in real time and stored as time series data, and finally the update frequency data of each node is output.

[0025] S102: Based on the node update frequency data and in combination with the node's network status, dynamically evaluate the node's change frequency, and associate the evaluation result with the node information to generate node change frequency evaluation data; First, key network performance indicators such as node delay, packet loss rate and bandwidth utilization are collected. The network status of each node is quantified by calculating the mean and standard deviation of these performance indicators. The network status data and node update frequency data are associated and processed. The two are matched according to the node unique identifier, and the matched data is feature extracted, such as using the maximum value, minimum value or change rate to analyze the change trend of the node. At the same time, combined with the time series data of the node update frequency, the change frequency of the node is dynamically evaluated using the time series analysis method. The specific operation is as follows: Assuming that a node has multiple update records in a fixed time window, it can be sorted according to the time when the update event occurs, the time interval between two adjacent updates can be extracted, and the length of these time intervals can be recorded to further analyze whether these time intervals show regular changes; by segmenting these time intervals, for example, according to the time granularity of hours, days, etc., the average value and maximum and minimum values ​​of the time interval in each time period can be counted to observe whether there is an obvious change trend; in addition, the rate of increase or decrease can be calculated according to the change of the time interval, and this can be used as the basis for dynamically evaluating the change frequency, and finally the change frequency result of the node in a specific time period is obtained, and the result is associated with the node information in combination with the network status information to generate the node change frequency evaluation data.

[0026] S103: Based on the node change frequency evaluation data and in combination with the node conflict probability during the data synchronization process, the node conflict detection frequency is adjusted, and the adjusted conflict detection frequency is recorded to generate conflict occurrence probability data; Based on the node change frequency evaluation data, the formula is used: ; Compute the probability of conflict between nodes during data synchronization ; in, Indicates the update change rate of the node, reflecting the activity of the node during the data synchronization process. This value comes from the node change frequency evaluation data, which is obtained by counting the number of changes of the node within a certain time window. For example, if a node has changed 5 times in the past minute, the change frequency is 5 times / minute. Indicates the frequency of conflict detection on the node, reflecting the density of node checking during data synchronization. This value comes from the conflict detection log of the node and can be obtained by analyzing the conflict detection operation records in each time period. For example, by extracting the number of detections of a node in the system log, the detection frequency of the node is calculated to be 10 times / second. It indicates the sensitivity of the probability of conflict to the change of detection frequency, and reflects the rate of change of the probability of conflict with the change of detection frequency. This parameter is usually determined through experiments. For example, representative nodes are selected as experimental objects, and the conflict probability data of these nodes under different conflict detection frequencies are collected. For example, the actual number of conflicts of nodes under detection frequencies such as 2 times / second, 5 times / second, and 10 times / second is recorded, and the corresponding conflict probability is calculated; then, these conflict probabilities and detection frequency data points are plotted into a change curve, and the trend of conflict probability with detection frequency is analyzed. The slope of the curve reflects the sensitivity of detection frequency to conflict probability, and thus it is estimated. Next, by adjusting the experimental conditions, the detection frequency and the network status in the node environment are gradually optimized to verify the Whether it is consistent, further fit the experimental curve to find the slope value that best suits the experimental data as the final Set the result, Indicates the reference detection frequency of a node, which is used to define the midpoint of the detection frequency. This value can be determined based on the statistical value of the system's historical data. For example, the average or median value of the detection frequency of all nodes in a system is used as the reference frequency. is the base of natural logarithms, Indicates the degree of deviation of the conflict detection frequency from the reference value, and is expressed by the sensitivity parameter The adjustment is used to control the impact of the detection frequency on the conflict probability. The operation result in the exponential term is calculated by directly substituting the parameter value.

[0027] For example, the update records of a node in the last minute are: update at 1 second, update at 12 seconds, and update at 30 seconds. In total, 3 updates occurred in 1 minute, and the node change frequency is 3 times / minute.

[0028] Conflict detection frequency :Acquired through log extraction. For example, in a system log, it is recorded that a node is detected 5 times within 1 second, so the conflict detection frequency is 5 times / second.

[0029] Sensitivity parameters :Determined by experiment. For example, through the collision probability experiment under different detection frequencies (such as 2 times / second, 5 times / second, 10 times / second), the data obtained are as follows: the probability is 0.2 at 2 times / second, 0.5 at 5 times / second, and 0.8 at 10 times / second. By fitting the curve of these data, we can get The value of .

[0030] Baseline detection frequency :Determined by the statistical value of historical data. For example, if the average detection frequency of a node in the past period of time is 5 times / second, then the benchmark value is set is 5.

[0031] Assume the change frequency of a node , conflict detection frequency times / second, sensitivity parameter , reference frequency . Substituting these parameters into the formula: ; The result shows that the probability of conflict for this node under the current conditions is 56.88%. According to the result of the conflict probability, judge whether the current detection frequency is reasonable. For example, if the conflict probability exceeds 50%, it is considered that the detection frequency may be too high, and it is necessary to appropriately reduce the detection frequency to reduce resource waste, and observe the change of the conflict probability; secondly, by setting adjustment rules, such as reducing the detection frequency by 1 time / second each time, re-collect the conflict occurrence data of the node, and calculate the corresponding new conflict probability after adjustment; repeat this process until the conflict probability is reduced to below the preset threshold (such as 30%), and then record the conflict detection frequency at this time as the adjusted detection frequency value; finally, store the adjusted detection frequency and the corresponding conflict probability as a set of associated data, and record it in the conflict probability data.

[0032] See also Figure 3 , the network status of each node in the conflict probability data is combined to evaluate the impact of conflicts during data synchronization, and the future synchronization delay of the target synchronization path is predicted based on the evaluation results. The specific steps for generating the delay trend analysis results are as follows: S201: Based on the conflict probability data, combined with the network status of each node, the synchronization delay record and the data size of the real-time synchronization task, the conflict impact of the current synchronization is evaluated to generate conflict impact evaluation data; The specific process of combining the network status, synchronization delay record and data size of each node in real-time synchronization task is as follows: for example, node A has a conflict probability of 45%. The network status data of the node includes an average delay of 50ms and a packet loss rate of 2%. The synchronization delay record shows that the delays of the last five synchronization tasks are 60ms, 70ms, 55ms, 65ms and 80ms respectively. These data are stored in the form of time series. The synchronization task data size is parsed from the task log to be 2MB, 3MB and 5MB respectively. In order to evaluate the impact of node A on the probability of conflict, firstly, according to the historical correlation between network status and synchronization delay, the fluctuation range of delay and packet loss rate is extracted, and it is cross-compared with the synchronization task size. It is found that when the task size exceeds 3MB, the average delay increases to 75ms and the packet loss rate increases to 4%. Secondly, by performing regression analysis on the growth trend of delay and packet loss rate, the impact of task size on the probability of conflict is quantified, and the change curve of the probability of conflict with the increase of delay is observed. In addition, we divide large tasks (such as 5MB tasks) into 2MB small tasks for synchronization, and measure the delay and packet loss rate data again. We find that the delay drops to 50ms, the packet loss rate recovers to 2%, and the adjusted conflict probability is recorded as 30%. Therefore, we can evaluate the impact of the conflict probability of node A, and conclude that block transmission can reduce the probability of conflict. At the same time, we store the adjusted conflict probability data as the node conflict impact evaluation result.

[0033] S202: Based on the conflict impact assessment data and in combination with the processing capability information of the nodes, the synchronization behavior of the nodes in the target path is predicted to generate a synchronization delay trend analysis result of the target path; Combined with the processing capacity information of the nodes, the specific process of predicting the synchronization behavior of the nodes in the target synchronization path is as follows: for example, the target path contains nodes B, C, and D, and the processing capacity data of node B is 60% CPU usage and 70% memory usage, while the CPU usage of nodes C and D are 80% and 50%, and the memory usage is 85% and 60% respectively. The distribution ratio of tasks in the path is obtained by the ratio of the task flow size and the historical processing capacity records of each node. For example, when the total task flow obtained from the task flow log is 10MB, combined with the historical records of nodes B, C, and D, their average processing rates under the same task conditions are 5MB / s, 4MB / s, and 6MB / s respectively. The average processing rate of each node is added to the total rate and the ratio is calculated, and the initial distribution ratio is 5:4:6, that is, node B is allocated 3MB, node C is allocated 2.67MB, and node D is allocated 4MB. The current processing capacity limit of the node is collected and obtained through the performance monitoring tool Prometheus. For example, the monitoring record of node C shows that when the processing task flow is greater than 4MB, the CPU usage exceeds 90%, resulting in a decrease in synchronization performance. Combining these data, by calculating the distribution ratio of tasks in the path and the performance threshold of each node, it is predicted that the delay of node C will increase to 150ms under the current distribution conditions, thus affecting the overall synchronization delay of the path. To optimize the path performance, the task allocation of node C is adjusted to 3MB, and the excess task traffic is reallocated to node D. The adjusted CPU usage and memory occupancy are recorded, and it is found that the delay of node C is reduced to 100ms, and the overall synchronization delay of the path is restored to within 120ms, thereby generating the adjusted target path synchronization delay trend analysis results.

[0034] S203: Based on the target path synchronization delay trend analysis result, combined with the historical delay records of each node and the current network status, the delay change trend of the target path is summarized and analyzed to generate a delay trend analysis result; Combining the historical delay records of each node and the current network status, the specific process of summarizing and analyzing the delay change trend of the target path is as follows: for example, the target path consists of nodes B, C, and D. The synchronization delay trend analysis results of node B show that the current average delay is 120ms, and the average delays of nodes C and D are 150ms and 100ms respectively. At the same time, the historical delay records show that the delay of node C fluctuates greatly when the task traffic exceeds 4MB, and the maximum delay reaches 200ms, while the delay fluctuation range of nodes B and D is relatively small. The current network status is obtained through the delay monitoring and monitoring tool Prometheus. For example, the current packet loss rate of node C is 4%, and the bandwidth utilization is 85%, while the packet loss rate of nodes B and D is 2%, and the bandwidth utilization is 70% and 60% respectively. By analyzing the above data, the historical delay records are matched with the current network status. For example, the packet loss rate and bandwidth utilization of node C are associated with the time period of delay fluctuation. It is found that the high packet loss rate and high bandwidth utilization of node C are strongly correlated with the delay growth. The delay data is further decomposed through segmented statistical methods. For example, the delay is divided into short-term fluctuations and long-term changes. By calculating the average value of short-term fluctuations and the trend slope of long-term changes, the variation law of the delay in the path is summarized. Finally, these summary results are integrated into the delay change trend data of the target path and output as the delay trend analysis results of the target path.

[0035] See also Figure 4 ,According to the delay trend analysis results, the corresponding synchronization path is selected for data transmission, and the current node load status is evaluated in real time during the data transmission process. The specific steps for generating path selection and load evaluation data are as follows: S301: Based on the delay trend analysis result, the current transmission quality of each synchronization path is measured and evaluated in real time, the quality parameters of the path are recorded, and the path transmission quality information is generated; First, you need to collect relevant parameters in the path, such as the delay, bandwidth utilization and packet loss rate of each path. You can use the performance monitoring tool Prometheus to collect data on the nodes and links in the path, record the data transmission delay between each node, and obtain the bandwidth utilization and packet loss rate of the current path in combination with the real-time network status. In the specific operation process, the delay is weighted and accumulated according to the node sequence of the path, the packet loss rate is calculated by recording the proportion of fixed data packet loss, and the bandwidth utilization is determined by the ratio of the actual transmission rate of the link to the maximum transmission capacity. The above measurement data is organized into structured information storage and associated with the unique identification information of the path. At the same time, the current transmission quality parameters of the path are recorded to generate the path transmission quality information.

[0036] S302: Based on the path transmission quality information and the network status, a synchronization quality score of each path is calculated by weighted combination, and the score result is associated with the path information to generate a path synchronization quality score; To calculate the synchronization quality score of each path, the formula is used: ; Get the synchronization quality score of the target path ; in, They respectively represent the weight parameters of latency, packet loss rate, and bandwidth utilization in path synchronization quality. The weight parameters are determined by actual application scenarios. For example, latency may have a greater weight in real-time communication applications, and bandwidth may have a greater weight in file transfer applications. The specific values ​​are adjusted through experiments or actual scenario requirements. For example, multiple groups of experiments with different weights are conducted to compare the degree of conformity between the scoring results and the actual synchronization performance, thereby determining the weight values. Indicates the average transmission delay of the path (unit: milliseconds), which is obtained through the monitoring tool Prometheus. For example, within a specified time window, multiple data transmission delays on the path are recorded and their average value is calculated. The delay data comes from the network communication time record between nodes. Assuming that in a certain path, the delays when transmitting test data packets are 10ms, 15ms, and 20ms respectively, the average path delay is: 15ms, Indicates the proportion of packet loss in data transmission on a path (unit: percentage). The network monitoring system counts the number of sent and received data packets within a fixed time, calculates the packet loss rate and takes the average value. Assuming that on a certain path, 1000 data packets are sent and 950 data packets are received within one minute, the packet loss rate is: , Indicates the average bandwidth usage of the path, which is the ratio of the actual bandwidth used by the path to the maximum available bandwidth of the path (unit: percentage). The actual bandwidth utilization data is obtained through the monitoring tool Prometheus, and the average value of the nodes in the path is calculated. Assuming that in a certain path, the actual bandwidth utilization is monitored to be 70%, 80%, and 90%, then the average bandwidth utilization of the path is: 80%, It is the maximum and minimum values ​​of the delay, indicating the maximum and minimum values ​​of the path delay in the historical measurement data. It is used to normalize the delay data. It is obtained through long-term monitoring of multiple path delay data and the maximum and minimum values ​​in the data range are selected. For example, by monitoring the paths in the past week, the peak and valley values ​​of the delays on different paths are recorded. It is the maximum and minimum values ​​of the packet loss rate, indicating the maximum and minimum values ​​of the path packet loss rate in the historical measurement data. It is used to normalize the packet loss rate data. It is obtained by counting the packet loss rate data of all paths in different time windows, for example, by extracting the network status monitoring records in the past month. It is the maximum and minimum values ​​of bandwidth utilization, indicating the maximum and minimum values ​​of path bandwidth utilization in historical measurement data. It is used to normalize bandwidth utilization data and determine the upper and lower limits by collecting bandwidth utilization monitoring data of different paths.

[0037] Assuming the average delay of the path , average packet loss rate , average bandwidth utilization , and its historical maximum and minimum values ​​are , , , , , The weight is , , .

[0038] Delay normalization: ; Packet loss rate normalization: ; Bandwidth utilization normalized: ; Calculate the overall score: ; The results show that the score represents the comprehensive synchronization quality of the path is 0.7811 (normalized score range is 0 to 1). The unique identifier of the path is used as the index, and the score result is stored in the path transmission quality database. At the same time, other key information of the path is associated, including the average delay, average packet loss rate, average bandwidth utilization and the timestamp of collecting these parameters, to ensure that the score result corresponds to the specific network status.

[0039] S303: Based on the path synchronization quality score, select the path with the highest synchronization score for data transmission, and monitor the load status of the node in real time during the transmission process, record the load change of the node and the transmission process, and generate path selection and load evaluation data; By sorting the path synchronization quality score data, the path with the highest score is selected and marked as the priority transmission path, and the unique identification information and related node list of the path are associated. During the data transmission process, the load status of each node in the path is monitored in real time. For example, the CPU usage, memory usage and network bandwidth utilization of each node are collected through the performance monitoring tool Prometheus, and these load data are recorded in the form of time series. In the specific operation, the load status data is continuously collected within a fixed time window, for example, the CPU and memory usage of the node are recorded every 5 seconds, the time series data is smoothed to eliminate short-term fluctuations, and the load peak interval is marked. If the load status of a node exceeds the preset threshold, for example, the CPU usage reaches 90%, the load balancing mechanism is triggered to reallocate part of the data traffic to the backup path with the second highest score, and the data traffic distribution and path performance changes before and after the load adjustment are recorded. Finally, the load change records of each node during the transmission process and the path selection data are combined to generate complete path selection and load evaluation data, which provides a basis for subsequent optimization of path transmission efficiency and improvement of system stability.

[0040] See also Figure 5 ,Based on the path selection and load evaluation data, monitor the load and bandwidth status of all nodes, dynamically adjust the allocation of data transmission tasks according to the load and bandwidth status of the nodes, and generate scheduling priority and resource allocation data. The specific steps are as follows: S401: Based on the path selection and load evaluation data, monitor and collect statistics on the real-time load and bandwidth status of all nodes to generate node load and bandwidth status; The CPU usage, memory usage, and bandwidth utilization of nodes are collected in real time through the performance monitoring tool Prometheus, and these data are stored in time series, for example, recording the resource utilization of nodes in units of 5 seconds. The specific process is: obtain the CPU usage of each node during task execution, for example, the CPU usage of node A is 65%, node B is 80%, and node C is 55%, and the memory usage is 70%, 85%, and 50% respectively, and monitor the bandwidth utilization at the same time. For example, the transmission rate of node A is 500Mbps, and the maximum bandwidth of the node is 1000Mbps, then the bandwidth utilization is 50%. The real-time data of all nodes are counted by time period, for example, the average load status and fluctuation range of each node are calculated every minute, and the time period of the load peak and trough is marked. For example, the CPU peak of node B occurs in the second minute, and the utilization rate reaches 90%. Combined with the statistical results, the resource status of the node is analyzed. For example, nodes with a load exceeding 80% are marked as high-resource utilization nodes, and the overall bandwidth utilization distribution of each node is further summarized, and finally the load and bandwidth status of all nodes are generated.

[0041] S402: dynamically allocate and adjust the data transmission tasks based on the node load and bandwidth conditions, refine and associate the node load and bandwidth change trends, and generate dynamic task allocation results; Refine and correlate the node load and bandwidth change trends, for example, by recording the CPU usage curve of the node during task execution, marking the load fluctuation range, and combining the time series changes of bandwidth utilization to partition the resource status of each node. Assuming that the CPU usage of node A is stable at around 65%, and the bandwidth utilization increases from 50% to 70%, while the CPU usage of node B fluctuates greatly, from 80% to 90%, and the bandwidth utilization increases from 75% to 85%, the analysis shows that the load pressure of node B is greater, and the tasks assigned to node B should be reduced first. In the process of dynamically adjusting task allocation, for example, assuming that the data volume of a task is 500MB, it needs to be allocated to three nodes for transmission. According to the current load status of the nodes, the initial allocation ratio is 200MB for node A, 200MB for node B, and 100MB for node C. However, after the adjustment, the allocated data volume of node B is reduced to 100MB, and the remaining data volume is redistributed to node A and node C, so that node A is allocated 250MB and node C is allocated 150MB, and the load changes after adjustment are monitored in real time. Through the dynamic optimization of the task allocation ratio, the dynamic task allocation result is finally generated, including the adjusted task distribution and the resource usage changes of each node.

[0042] S403: Based on the dynamic task allocation result, the scheduling priority of each data transmission task is reallocated, and the scheduling priority and resource allocation data are generated in combination with the current load and bandwidth resource allocation situation; First, sort the data in the dynamic task allocation results. For example, tasks T1, T2, and T3 correspond to nodes A, B, and C, respectively. The data volume of task T1 is 500MB, task T2 is 300MB, and task T3 is 200MB. According to the data volume of the tasks and the load status of the nodes, the analysis is performed. Assuming that the bandwidth utilization of node A is 50%, node B is 85%, and node C is 60%, the priority of task T2 is reduced and adjusted to the last transmission based on the node load and bandwidth conditions. Task T1 is adjusted to the highest priority transmission because the bandwidth utilization of the corresponding node is the lowest and the data volume is the largest. The transmission time and node status changes are recorded during the execution process of task scheduling. For example, the time required for task T1 to complete is 2 minutes, during which the CPU usage of node A fluctuates between 60% and 70%, and the bandwidth utilization is stable at 55%. The time required for task T3 to complete is 1 minute, and the bandwidth utilization of node C increases from 60% to 65%. By combining the scheduling results and node status, the scheduling priority and resource allocation data are finally generated.

[0043] See also Figure 6 ,Based on the scheduling priority and resource allocation data, the data synchronization tasks of each priority are assigned to the storage nodes of the corresponding level in the cloud platform. The specific steps for generating cloud storage level synchronization data are as follows: S501: Based on the scheduling priority and resource allocation data, the data synchronization tasks of each priority are associated and matched with the storage nodes of the corresponding level in the cloud platform, the corresponding relationship between the priority and the storage node is analyzed, and the mapping result between the priority and the storage node is generated according to the analysis result; First, sort the tasks according to their priority, for example, high-priority tasks are marked as first-level priority, and low-priority tasks are marked as second-level and third-level priority, respectively. Then extract the data volume of the tasks and the corresponding node requirements. For example, the data volume of high-priority tasks is 200GB, and the corresponding node storage hierarchy requirement is a high-speed storage node (such as the SSD layer). The data volume of medium-priority tasks is 500GB, and the corresponding node is a medium-speed storage node (such as the HDD layer). By analyzing the matching relationship between task priority and storage node performance, for example, based on the read and write rate and current occupancy rate of the storage node, select nodes that meet the task requirements, match high-priority tasks to nodes with high read and write rates and sufficient storage space, and record the node hierarchy information and task allocation relationship. Combined with the task priority and storage node matching, the priority and storage node mapping results are finally generated to provide a shard storage basis for subsequent data synchronization tasks.

[0044] S502: Based on the mapping result between the priority and the storage node, data sharding is performed on the data synchronization content of the task according to the priority order, and the storage location of the sharded data is allocated according to the storage node capacity to generate priority sharding and storage location information; The storage location of the shard data is allocated according to the capacity of the storage node. For example, if the data volume of a high-priority task is 200GB, the data needs to be divided into several fragments, and the size of each fragment is controlled within 50GB. The task data is divided into 4 fragments and marked with serial numbers through the sharding algorithm. At the same time, the capacity of the storage node is monitored in real time. For example, the available capacity of the mapped high-speed storage nodes is 100GB, 60GB and 80GB respectively. According to the capacity priority allocation rule, the 4 shards of the task are stored in these nodes respectively, and the storage location and storage node information corresponding to each shard are recorded. For medium-priority and low-priority tasks, they are processed according to the same sharding rules. For example, the 500GB data of the medium-priority task is divided into 10 fragments, which are stored in the medium-speed storage node in turn, and the location information of the fragments and the task priority are recorded, and finally the priority fragments and storage location information are generated.

[0045] S503: Based on the priority sharding and storage location information, the sharding data is synchronously transmitted to the corresponding storage node, and the hierarchical storage structure information after the transmission is completed is recorded to generate cloud storage hierarchical synchronization data; Use data transfer tools such as Rclone to monitor the status of each shard during the transmission process, such as recording the transmission rate, transmission time, and data verification results of the shard, and record the node hierarchical information of the completed data transmission into the hierarchical storage structure. For example, the data transmission rate of the high-priority shard of the task is 200MB / s, and the completion time is 4 minutes. After the storage is completed, the storage path and hierarchical identification of the shard in the high-speed storage node are recorded; the data transmission rate of the shard of the medium-priority task is 100MB / s, and the completion time is 5 minutes, which is recorded in the hierarchical storage structure of the medium-speed storage node. Monitor the data integrity of all shards during the transmission process, such as performing MD5 verification on each shard to ensure that the shard data is consistent with the original data, and record the verification results. Combine all the shard data and storage node information that have been transferred, and finally generate cloud storage hierarchical synchronization data to provide support for storage management and data scheduling.

[0046] See also Figure 7 , a cloud-based data synchronization system, the system includes: The conflict detection and adjustment module collects the node set in the cloud environment, calculates the probability of conflict for each node during data synchronization, adjusts the frequency of data synchronization conflict detection, and obtains conflict probability data; The delay prediction module combines the network status of each node in the conflict probability data, evaluates the impact of conflicts during data synchronization, predicts the future synchronization delay of the target synchronization path, and generates delay trend analysis results; The path selection and load assessment module selects the corresponding synchronization path for data transmission according to the delay trend analysis results, evaluates the current node load status in real time during the data transmission process, and generates path selection and load assessment data; The resource scheduling optimization module monitors the load and bandwidth status of all nodes based on path selection and load evaluation data, dynamically adjusts the allocation of data transmission tasks according to the load and bandwidth status of the nodes, and generates scheduling priority and resource allocation data; The cloud storage hierarchical synchronization module allocates data synchronization tasks of each priority level to storage nodes of the corresponding level in the cloud platform based on scheduling priorities and resource allocation data, and generates cloud storage hierarchical synchronization data.

[0047] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0048] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0049] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0050] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0051] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0052] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0053] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0054] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0055] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0056] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A cloud-based data synchronization method, characterized in that: The following steps are involved: S1: Collect the node set in the cloud environment, the probability of conflict in each node during data synchronization, and adjust the frequency of data synchronization conflict detection to obtain conflict probability data; S2: combining the network status of each node in the conflict probability data, evaluating the impact of the conflict during data synchronization, predicting the future synchronization delay of the target synchronization path according to the evaluation result, and generating a delay trend analysis result; S3: According to the delay trend analysis result, select a corresponding synchronization path for data transmission, and evaluate the current node load status in real time during the data transmission process to generate path selection and load evaluation data; S4: Based on the path selection and load evaluation data, monitor the load and bandwidth status of all nodes, dynamically adjust the allocation of data transmission tasks according to the load and bandwidth status of the nodes, and generate scheduling priority and resource allocation data; S5: Based on the scheduling priority and resource allocation data, the data synchronization tasks of each priority level are allocated to storage nodes of corresponding levels in the cloud platform to generate cloud storage level synchronization data.

2. The cloud-based data synchronization method according to claim 1, characterized in that: The conflict probability data includes node update frequency, network delay, and node load status. The delay trend analysis results include the delay value of the target synchronization path, the influence coefficient of the data transmission size on the path, and the weighted value of the node processing capability to the delay. The path selection and load evaluation data include the path real-time synchronization quality score, the node load status score, and the bandwidth utilization of the transmission path. The scheduling priority and resource allocation data include the task scheduling priority weight, the node bandwidth allocation ratio, and the node resource utilization after task allocation. The cloud storage level synchronization data includes cache task sharding data, distributed storage task processing data, and storage task archiving data.

3. The cloud-based data synchronization method according to claim 1, characterized in that: Collect the node set in the cloud environment, the probability of conflict in each node during data synchronization, and adjust the frequency of data synchronization conflict detection. The specific steps to obtain the conflict probability data are as follows: S101: Based on the node set in the cloud environment, statistically analyze the update frequency of the nodes, and record the update timestamp of each node in real time to generate node update frequency data; S102: Based on the node update frequency data and in combination with the network status of the node, dynamically evaluate the change frequency of the node, and associate the evaluation result with the node information to generate node change frequency evaluation data; S103: Based on the node change frequency evaluation data and in combination with the conflict probability of the node during data synchronization, the conflict detection frequency of the node is adjusted, and the adjusted conflict detection frequency is recorded to generate conflict occurrence probability data.

4. The cloud-based data synchronization method according to claim 3, characterized in that: Based on the node change frequency evaluation data, the formula is used: ； Compute the probability of conflict between nodes during data synchronization ; in, Indicates the update change rate of the node, Indicates the frequency of node conflict detection. It represents the sensitivity coefficient of the probability of conflict occurrence to the change of detection frequency, represents the reference detection frequency of the node, is the base of natural logarithms.

5. The cloud-based data synchronization method according to claim 1, characterized in that: The specific steps of combining the network status of each node in the conflict probability data, evaluating the impact of the conflict during data synchronization, predicting the future synchronization delay of the target synchronization path according to the evaluation result, and generating the delay trend analysis result are as follows: S201: Based on the conflict probability data, combined with the network status of each node, the synchronization delay record and the data size of the real-time synchronization task, the conflict impact of the current synchronization is evaluated to generate conflict impact evaluation data; S202: Based on the conflict impact assessment data and in combination with the processing capability information of the nodes, the synchronization behavior of the nodes in the target path is predicted to generate a target path synchronization delay trend analysis result; S203: Based on the target path synchronization delay trend analysis result, combined with the historical delay records of each node and the current network status, the delay change trend of the target path is summarized and analyzed to generate a delay trend analysis result.

6. The cloud-based data synchronization method according to claim 1, characterized in that: According to the delay trend analysis result, the corresponding synchronization path is selected for data transmission, and the current node load status is evaluated in real time during the data transmission process. The specific steps for generating path selection and load evaluation data are as follows: S301: Based on the delay trend analysis result, the current transmission quality of each synchronization path is measured and evaluated in real time, the quality parameters of the path are recorded, and the path transmission quality information is generated; S302: Based on the path transmission quality information and the network status, a synchronization quality score of each path is calculated by weighted combination, and the scoring result is associated with the path information to generate a path synchronization quality score; S303: Based on the path synchronization quality score, select the path with the highest synchronization score for data transmission, and monitor the load status of the node during the transmission process in real time, record the load change of the node and the transmission process, and generate path selection and load evaluation data.

7. The cloud-based data synchronization method according to claim 6, characterized in that: To calculate the synchronization quality score of each path, the formula is used: ； Get the synchronization quality score of the target path ; in, They represent the weight parameters of delay, packet loss rate and bandwidth utilization in path synchronization quality, represents the average transmission delay of the path, Indicates the proportion of packet loss in data transmission along the path Indicates the average bandwidth usage of the path, are the maximum and minimum values ​​of the delay, are the maximum and minimum values ​​of the packet loss rate, These are the maximum and minimum values ​​of bandwidth utilization.

8. The cloud-based data synchronization method according to claim 1, characterized in that: Based on the path selection and load evaluation data, the load and bandwidth status of all nodes are monitored, and the allocation of data transmission tasks is dynamically adjusted according to the load and bandwidth status of the nodes. The specific steps of generating scheduling priority and resource allocation data are as follows: S401: Based on the path selection and load evaluation data, monitor and collect statistics on the real-time load and bandwidth status of all nodes to generate node load and bandwidth status; S402: Based on the node load and bandwidth status, dynamically allocate and adjust the data transmission task, refine and associate the node load and bandwidth change trend, and generate a dynamic task allocation result; S403: Based on the dynamic task allocation result, the scheduling priority of each data transmission task is reallocated, and the scheduling priority and resource allocation data are generated in combination with the current load and bandwidth resource allocation situation.

9. The cloud-based data synchronization method according to claim 1, characterized in that: Based on the scheduling priority and resource allocation data, the data synchronization tasks of each priority are allocated to the storage nodes of the corresponding level in the cloud platform. The specific steps of generating cloud storage level synchronization data are as follows: S501: Based on the scheduling priority and resource allocation data, the data synchronization tasks of each priority are associated and matched with the storage nodes of the corresponding level in the cloud platform, the corresponding relationship between the priority and the storage node is analyzed, and the mapping result between the priority and the storage node is generated according to the analysis result; S502: Based on the priority and storage node mapping result, data sharding is performed on the data synchronization content of the task according to the priority order, and the storage location of the sharded data is allocated according to the storage node capacity to generate priority sharding and storage location information; S503: Based on the priority sharding and storage location information, the sharding data is synchronously transmitted to the corresponding storage node, and the hierarchical storage structure information after the transmission is completed is recorded to generate cloud storage hierarchical synchronization data.

10. A cloud-based data synchronization system, characterized in that: According to any one of claims 1 to 9, the cloud-based data synchronization method is implemented, and the system comprises: The conflict detection and adjustment module collects the node set in the cloud environment, calculates the probability of conflict for each node during data synchronization, adjusts the frequency of data synchronization conflict detection, and obtains conflict probability data; The delay prediction module combines the network status of each node in the conflict probability data, evaluates the impact of the conflict during data synchronization, predicts the future synchronization delay of the target synchronization path, and generates a delay trend analysis result; The path selection and load evaluation module selects the corresponding synchronization path for data transmission according to the delay trend analysis result, evaluates the current node load status in real time during the data transmission process, and generates path selection and load evaluation data; The resource scheduling optimization module monitors the load and bandwidth status of all nodes based on the path selection and load evaluation data, dynamically adjusts the allocation of data transmission tasks according to the load and bandwidth status of the nodes, and generates scheduling priority and resource allocation data; The cloud storage hierarchical synchronization module allocates data synchronization tasks of each priority level to storage nodes of corresponding levels in the cloud platform based on the scheduling priority and resource allocation data, and generates cloud storage hierarchical synchronization data.

Images