Adaptive database consistency management method and system

Abstract

The invention discloses a self-adaptive database consistency management method and system. The management method comprises the following steps: collecting performance data; carrying out analysis and decision making on the performance data to obtain an analysis result; performing consistency level adjustment on the analysis result; and carrying out threshold feedback adjustment. And a more flexible and efficient management method is provided for the distributed database system in a complex and dynamic edge environment.

Description

Technical Field

[0001] The present invention relates to the field of Internet and edge computing technology, and in particular to an adaptive database consistency management method and system. Background Art

[0002] With the rapid development of the Internet of Things (IoT) and edge computing technology, more and more computing tasks are transferred from central data centers to edge nodes of the network to improve the real-time and response speed of data processing. However, since edge nodes usually have limited computing and storage resources and unstable network connections, this poses new challenges to the consistency management of distributed database systems.

[0003] Traditional distributed database systems rely on strong consistency models, such as Paxos and Raft, which ensure data consistency through extensive communication between all nodes. However, this approach may lead to high latency and performance bottlenecks in resource-constrained edge computing environments, and cannot meet the requirements of low latency and high availability. Therefore, in edge computing environments, a solution is needed that can dynamically adjust the consistency strategy according to real-time network conditions and system load to balance the consistency and performance of the system.

[0004] In existing distributed data storage systems, ensuring data consistency usually relies on fixed strong consistency protocols such as Paxos or Raft. These protocols ensure that all data copies in the system remain consistent by conducting extensive communication between all participating nodes. However, although this approach can provide highly reliable consistency, in practical applications, especially in environments with high network latency and frequent node failures, it will lead to high system overhead and performance bottlenecks, especially in edge computing environments, where these problems are more prominent.

[0005] In traditional distributed database systems, fixed consistency levels are usually used, which means that when deploying the system, developers need to pre-set consistency parameters, such as replication factor and number of response nodes. This configuration is static during system operation and cannot be dynamically adjusted according to the actual network status and system load. Since the network conditions and resource usage in edge computing environments are dynamically changing, fixed consistency configurations often cannot meet the needs of all application scenarios, resulting in an inability to strike a balance between consistency and performance in some cases.

[0006] Disadvantages of existing technology:

[0007] 1. Fixed consistency configuration lacks flexibility: In existing distributed database systems, fixed strong consistency configuration is usually used. This means that consistency parameters need to be set in advance when the system is deployed, and this configuration cannot be adjusted dynamically during system operation. Since the network conditions and resource usage in edge computing environments are dynamically changing, fixed consistency configuration cannot strike a balance between consistency and performance in some cases, resulting in higher latency and resource waste when the system is under high load or the network is unstable.

[0008] 2. Network latency leads to performance degradation: Traditional consistency models (such as Paxos and Raft) require extensive communication between multiple nodes to ensure data consistency. This approach will significantly increase the delay of data synchronization in edge computing environments with high or unstable network latency, thereby reducing the overall performance of the system and failing to meet the needs of real-time data processing.

[0009] 3. Low resource utilization efficiency: In an edge computing environment with limited resources, fixed consistency configuration leads to inefficient resource utilization. For example, when the network conditions are good, the system may not need to maintain a high consistency level, but due to configuration limitations, a large amount of resources will still be consumed to maintain unnecessary strong consistency. This inflexible resource usage makes it difficult for the system to operate efficiently under high load. Summary of the invention

[0010] In view of the above problems, the present invention is proposed to provide an adaptive database consistency management method and system that overcomes the above problems or at least partially solves the above problems.

[0011] According to one aspect of the present invention, there is provided an adaptive database consistency management method, the management method comprising:

[0012] Collect performance data;

[0013] Analyze and make decisions on the performance data to obtain analysis results;

[0014] Adjusting the consistency level of the analysis results;

[0015] Make threshold feedback adjustments.

[0016] Optionally, the collecting performance data specifically includes: regularly collecting key performance indicators of network delay and node load.

[0017] Optionally, the analyzing and making decisions on the performance data to obtain the analysis results specifically includes:

[0018] The consistency adjustment module uses machine learning algorithms to predict the optimal consistency level based on the collected data.

[0019] Optionally, the adjusting the consistency level of the analysis result specifically includes:

[0020] Based on the analysis results, the coordinating node sends consistency level adjustment instructions to the replica nodes, and the replica nodes adjust the data synchronization strategy accordingly.

[0021] Optionally, the threshold feedback adjustment specifically includes:

[0022] The system sets a threshold for consistency adjustment, and consistency adjustment is triggered only when the performance indicator exceeds the threshold.

[0023] Optionally, the system sets a threshold for consistency adjustment, and triggers consistency adjustment only when the performance indicator exceeds the threshold. Specifically, the system sets a threshold for ...

[0024] The formula is used to evaluate the probability P of the expected consistency that the system can achieve under the current network delay L;

[0025] The parameters alpha, beta, gamma and threshold T are optimized based on historical performance data to adjust the sensitivity and response threshold of the model.

[0026] The present invention also provides an adaptive database consistency management system, which applies the above-mentioned adaptive database consistency management method, and the management system includes:

[0027] Performance monitoring module, used to collect performance data;

[0028] An analysis and decision module, used to analyze and make decisions on the performance data to obtain analysis results;

[0029] A consistency adjustment module, used for adjusting the consistency level of the analysis result;

[0030] The feedback adjustment module is used to perform threshold feedback adjustment.

[0031] The present invention provides an adaptive database consistency management method and system, the management method comprising: collecting performance data; analyzing and making decisions on the performance data to obtain analysis results; adjusting the consistency level of the analysis results; and adjusting the threshold feedback. It provides a more flexible and efficient solution for distributed database systems in complex and dynamic edge environments.

[0032] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0034] Figure 1 A flowchart of an adaptive database consistency management method provided by an embodiment of the present invention;

[0035] Figure 2 A block diagram of the composition of an adaptive database consistency management system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0037] The terms "comprises" and "having" and any variations thereof in the description embodiments, claims and drawings of the present invention are intended to cover non-exclusive inclusions, for example, including a series of steps or units.

[0038] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0039] The present invention provides an adaptive database consistency management method suitable for edge computing environment, which can dynamically adjust the consistency level according to the real-time status of the system (such as network conditions, node load, etc.) to optimize system performance and resource utilization efficiency.

[0040] like Figure 1 As shown, an adaptive database consistency management method includes: collecting performance data; analyzing and making decisions on the performance data to obtain analysis results; adjusting the consistency level of the analysis results; and performing threshold feedback adjustment.

[0041] Performance data collection: The performance monitoring module regularly collects key performance indicators such as network latency and node load.

[0042] Data analysis and decision-making: The consistency adjustment module uses machine learning algorithms to predict the optimal consistency level based on the collected data, reducing real-time computing requirements.

[0043] Consistency level adjustment: Based on the analysis results, the coordinating node sends consistency level adjustment instructions to the replica nodes, and the replica nodes adjust the data synchronization strategy accordingly.

[0044] Threshold feedback adjustment: The system sets thresholds for consistency adjustment, and triggers consistency adjustment only when performance indicators exceed these thresholds, reducing frequent and ineffective adjustments.

[0045] By formula To evaluate the possibility P of the expected consistency that the system can achieve under the current network delay L.

[0046] The parameter α is a scaling factor used to adjust P c It essentially sets an upper limit on the probability of achieving the desired level of consistency, ensuring that P c Keep it within a realistic range, usually between 0 and 1.

[0047] The parameter β controls P c The sensitivity to changes in delay L. A higher value of β means that as the delay increases, P c The decline is steeper. -β·L The part represents exponential decay; as the delay L increases, the c The impact of is exponentially smaller, reflecting that higher latency generally reduces consistency in distributed systems.

[0048] The parameter γ controls how fast the consistency probability changes relative to the delay threshold T. -γ·(L-T) The term induces a logistic function-like transition that helps model the sensitivity of the consistency probability with respect to the threshold T.

[0049] T is the latency threshold, beyond which it becomes significantly more difficult to maintain the required consistency level. When the latency L exceeds T, the formula reflects that P c A sharp decline indicates that the conditions for maintaining consistency have deteriorated.

[0050] In the formula, the LT part takes into account the difference between the delay L and the preset delay threshold T, which is used to determine whether the consistency level needs to be adjusted. The logistic function behavior is partially provided, which is very useful for smoothing the transition of simulated probability from one plateau to another with some input variable (LT in this case). This facilitates a more realistic adjustment of the coherence level as the simulated delay approaches and exceeds a critical threshold T.

[0051] The parameters α, β, γ and threshold T are optimized based on historical performance data and are used to adjust the sensitivity and reaction threshold of the model. The α, β, γ parameters help the model adjust the sensitivity to latency and the speed of consistency level adjustment. Through machine learning algorithms, the parameters are learned and optimized from system operation data to ensure the accuracy and practicality of the model.

[0052] By formula Calculate the average delay D of the entire system, where L i is the delay of the ith node, and δ is a coefficient used to adjust the overall delay sensitivity.

[0053] Delay calculation helps the system determine the current network status and decide whether the consistency strategy needs to be adjusted to adapt to the network status.

[0054] The scaling factor δ is used to adjust the influence of latency on consistency level decisions. Higher values ​​of δ mean that the system is more sensitive to latency, resulting in more frequent consistency level adjustments.

[0055] By applying mathematical models, the system is able to:

[0056] 1. Dynamic adaptation: Dynamically adjust the consistency level according to the real-time network status and node performance, and optimize the data synchronization strategy.

[0057] 2. Balance performance and consistency: Ensure that system performance is maximized without sacrificing data consistency.

[0058] 3. Reduce resource waste: Avoid over-synchronizing data when network conditions are good, save resources, and maintain necessary data consistency when network conditions are poor.

[0059] like Figure 2 As shown, an adaptive database consistency management system includes:

[0060] Coordinating node: responsible for monitoring the real-time status of the system (such as network latency, node load) and dynamically adjusting the consistency level of the database based on these statuses.

[0061] Replica node: stores replica data of the database and works with the coordination node to respond to consistency adjustment requests.

[0062] Performance monitoring module: responsible for real-time collection of system performance parameters such as network delay and node load.

[0063] Consistency adjustment module: Calculates and adjusts the consistency level based on the data provided by the performance monitoring module.

[0064] Beneficial effects:

[0065] Dynamic consistency level adjustment mechanism: It can dynamically adjust the consistency level based on real-time monitoring of network latency, node load and other performance data. This mechanism allows the system to adaptively optimize performance and resource utilization while ensuring data consistency, which is particularly suitable for edge computing environments where network conditions and system loads change frequently.

[0066] Application of performance monitoring and data analysis module: The invention includes an efficient performance monitoring module to collect key system operation indicators, such as network latency and node load. Combined with the data analysis module, the collected data is analyzed through advanced data processing algorithms (such as machine learning algorithms) to predict the optimal consistency level of the system. This is the basis for realizing the dynamic adjustment mechanism.

[0067] Threshold feedback control mechanism: In order to reduce frequent invalid adjustments caused by short-term fluctuations in network conditions, the present invention adopts a threshold feedback control mechanism. Only when the monitored performance index exceeds the preset threshold, the system will adjust the consistency level. This control mechanism improves the stability of the system and the effectiveness of the adjustment.

[0068] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An adaptive database consistency management method, characterized in that: The management method comprises: Collect performance data; Analyze and make decisions on the performance data to obtain analysis results; Adjusting the consistency level of the analysis results; Make threshold feedback adjustments.

2. The method for adaptive database consistency management according to claim 1, characterized in that: The collection of performance data specifically includes: regularly collecting key performance indicators of network delay and node load.

3. The adaptive database consistency management method according to claim 1, characterized in that: The analyzing and making decisions on the performance data to obtain the analysis results specifically includes: The consistency adjustment module uses machine learning algorithms to predict the optimal consistency level based on the collected data.

4. The method for adaptive database consistency management according to claim 1, characterized in that: The adjusting the consistency level of the analysis result specifically includes: Based on the analysis results, the coordinating node sends consistency level adjustment instructions to the replica nodes, and the replica nodes adjust the data synchronization strategy accordingly.

5. The method for adaptive database consistency management according to claim 1, characterized in that: The threshold feedback adjustment specifically includes: The system sets a threshold for consistency adjustment, and consistency adjustment is triggered only when the performance indicator exceeds the threshold.

6. The adaptive database consistency management method according to claim 5, characterized in that: The system sets a threshold for consistency adjustment, and consistency adjustment is triggered only when the performance index exceeds the threshold. Specifically, it includes: The formula is used to evaluate the probability P of the expected consistency that the system can achieve under the current network delay L; The parameters alpha, beta, gamma and threshold T are optimized based on historical performance data to adjust the sensitivity and response threshold of the model.

7. An adaptive database consistency management system, applying an adaptive database consistency management method according to any one of claims 1 to 6, characterized in that: The management system comprises: Performance monitoring module, used to collect performance data; An analysis and decision module, used to analyze and make decisions on the performance data to obtain analysis results; A consistency adjustment module, used for adjusting the consistency level of the analysis result; The feedback adjustment module is used to perform threshold feedback adjustment.

Images